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TheVirtualization Express: 
From Desktops to Databases 


To have been eager to jump on the 
virtualization train in recent years. The drive to 
virtualize has moved swiftly from desktop to server to 
applications. Now there's virtual desktop infrastructure 
(VDI). However, the industry has been slow to virtu- 
alize databases. Let's take a look at where the virtualiza- 
tion express has been so that we can prepare ourselves 
to ponder the future of database virtualization. 


Desktop and Server Virtualization 
Virtualization started on the desktop. With VMware 
Workstation, you could easily build multiple test 
environments without needing to reimage physical 
systems. From there, virtualization grew to the server 
level. VMware introduced its bare-metal hypervisor as 
a part of the VMware ESX server line. This, in com- 
bination with a new wave of multi-core x64 processors 
from Intel and AMD, gave server virtualization the 
performance and scalability needed to handle server 
consolidation for all types of production workloads. 
Microsoft was a latecomer to server virtualization, but 
the Hyper-V 2.0 virtualization platform that ships with 
Windows Server 2008 R2 has finally put it in a position 
comparable with VMware's. 


Application Virtualization 

The next wave in virtualization came at the application 
level. SoftGrid, which was later acquired by Microsoft 
and renamed Microsoft Application Virtualization 
(App-V), pioneered in this space. Application virtu- 
alization takes place between the application and the 
host OS. The application interacts with a copy of the 
system registry and file system. The level of interaction 
with the physical file system is usually configurable. 
Application virtualization is primarily intended for 
desktop applications because it facilitates central man- 
agement and application deployment. It isn't intended 
for server applications such as SQL Server or Exchange 
Server. Nowadays there are two basic types of virtual- 
ized applications: one that uses an agent on the host 
system, like Microsoft's App-V, and one that rolls the 
virtualization support into the executable and runs 
with no agent, like VMware ThinApp. In case you were 
wondering, yes, you can combine either desktop or 
server virtualization with application virtualization. 


Virtual Desktop Infrastructure 
The latest evolution in virtualization is VDI. Here 
desktop systems use the Remote Desktop Protocol 
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(RDP) or another remote display protocol to con- 
nect to a virtual machine (VM) running the appro- 
priate desktop OS on a backend Hyper-V or ESX 
server. Typically, a connection broker sits between the 
client systems and the back-end virtualization server 
directing the client requests to the correct VM. VDI 
provides centralized desktop management but also has 
high infrastructure requirements. 


Database Virtualization 

Today, database virtualization is done (if it's done 
at all) at the server level. However, the real future 
in database virtualization might well be in virtual- 
izing the database itself. In this scenario, the server 
might or might not be virtualized. Instead, the 
virtualization would take place at the database level; 
the database could be moved between different SQL 
Server instances in response to changing workloads. 
As the database is moved, the other database server 
elements, such as server properties, logins, permis- 
sions, files, and logs, could be moved right along with 
the database. When virtualization takes place at the 
database level and not at the server level, the databases 
would be transparently moved between servers with 
no downtime for the applications and end users. The 
potential advantages of this situation include more 
granular control over the workload that's carried by 
each server, as well as more control over the response 
time that's delivered by the applications using each 
database. Under conditions of low utilization indi- 
vidual databases or groups of databases could be 
moved to servers with lower capacities, and under 
higher workloads databases could be moved to servers 
with higher capacities. The virtualization itself might 
be managed by SQL Server using something like the 
Resource Governor that we see today. 

Whether this type of database virtualization will 
come to pass is anyone's guess, but it's certainly 
within the realm of possibility. In any event, clearly 
virtualization technology is not only here to stay but 
also growing in new directions, and it's enabling more 
efficient operations as well as providing a foundation 
for the dynamic data center—even for applications 
like SQL Server, which not so long ago many people 
believed couldn't be virtualized. Do you have any 
thoughts on virtualization and SQL Server? Are you 
looking forward to database virtualization? Drop me 
a line at letters@sqlmag.com. SOL) 

InstantDoc ID 125267 


Michael Otey 


(motey@ sqlmag.com) is technical director 
for Windows IT Pro and SQL Server 
Magazine and author of Microsoft SQL Server 
2008 New Features (Osborne/McGraw-Hill). 


July 2010 5 


SnarePointPro 


SharePointPro Connections Subscribe FREE to the only 
provides guidance to help align the technology with m ag azine dedicated to all 
business requirements. You'll get real-world advice from things Share Poin t. 


professionals and peers who share their experience 


administering and developing in SharePoint. 


Inaugural Issue of New Magazine fr eios and IT Pros 


SharePointPro Connections is the independent voice on MZ Z 
SharePoint technology; expert authors provide readers Shar [e Do Ol nt 
with the field-tested information they need to enable 


www.sharepointproconnections.com - eee 


content and image management, collaboration, and 


workflow solutions tailored to their business needs. 


ing to NI Jus 2919 
yst Opers Cin Prapare 


Upcoming articles include: Ie ime p^ 
| — jee epanungEc Mijaz zs 
Mi i i Creati ce Business, pr 
Migrating to SharePoint 201 0 | Apps in Sherspol 1200) p EDITORIAL/ 


STEVE FOX 
In with the New 


e Using SharePoint with Visual Studio 


* Getting more out of SharePoint 


ORDER YOUR FREE SUBSCRIPTION TODAY! 
sharepointproconnections.com/go/subscribetoday 


Use PowerShell to Generate 


HTML Reports 


efore I get started, I first must give credit 

where credit is due. My first look at Power- 

Shell was at a Professional Association for 
SQL Server (PASS) presentation by Allen White 
called ^Using PowerShell to Get the Most Out of SQL 
Server." During that presentation, he demonstrated 
how to use PowerShell to generate HTML files. I've 
simply implemented what he demonstrated. 

Where I work, we use custom jobs to monitor our 
environment and generate HTML reports. There are a 
large number of SQL Server machines, so we automate 
as many monitoring jobs as we can. For example, we 
have a job that runs daily to check for any databases 
that haven't been backed up. We generate an HTML 
file that lists any offenders so that a DBA can investi- 
gate why the database isn't getting backed up. 

In SQL Server 2000, we generated the HTML file 
using sp makewebtask. This stored procedure has been 
deprecated, so when we migrated our central reporting 
server to SOL Server 2008, we needed to find another 
solution. We decided to replace the sp makewebtask 
step in our monitoring jobs with a PowerShell step. 
The new step runs the same T-SQL code as the old, 
but instead of using sp makewebtask to generate the 
HTML file, it uses PowerShell's ConvertTo-HTML 
cmdlet. 

Listing 1 shows the PowerShell step that we now 
include in all our monitoring jobs. (You can down- 
load this code by going to www.sglmag.com, entering 
125232 in the InstantDoc ID text box, clicking Go, 
then clicking the Download the Code Here button.) The 
code begins by defining five variables and setting up 
the HTML formatting. The variables, which callout 
A shows, specify the T-SQL command to execute, the 
SQL Server instance and database against which to run 
that command, and the title and location of the HTML 
report that will contain the command’s results. 

Next, the code creates a string variable that contains 
all the connection details. The code uses the variable to 
connect to the data source, then executes the T-SQL 
command specified in callout A against it. After the 
command's results are put into an ADO.NET DataSet 
object, the connection is closed. 

Finally, the code returns all the rows from the query, 
piping them to the ConvertTo-HTML cmdlet. This 
cmdlet formats the results into an HTML page, which 
is piped to the output file. 

The results in the HTML page essentially look 
the same as the results you'd receive if you ran the 
T-SQL command in Query Analyzer with the output 
set to Results to Grid. Figure 1 shows sample results 
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ISTING I: The PowerShell Ste 
(A) ## Define the variables. 


## SQL instance name 

$dataSource = "localhost" 

## Database name 

$database = "DBAMonitoring" 

## T-SQL command to execute 
$sqlCommand = "exec usp MyReport" 
## Title of the HTML page 
$TableHeader - "My SQL Report" 
## Location of the HTML report 
$OutputFile = "C:\MyReport.htm" 


Adam Moericke 


## Set the HTML formatting. 

$a = @" 

<style> 

BODY{background-color:white; } 

TABLE{border-width: 1px; 
border-style: solid; 
border-color: black; 
border-collapse: collapse; 


ORE on the WEB 


Download the code at InstantDoc 
IDs 125232 and 125247 


) 

TH{border-width: 1px; 
padding: Spx; 
border-style: solid; 
border-color: black; 
background-color: CØCØCØ 


b 

TD{border-width: 1px; 
padding: 0px; 
border-style: solid; 
border-color: black; 
background-color:white 


</style> 
"@ 


$body = Q" 
<p style="font-size: 25px; 
family: calibri; 
color: #ff9199"> 
$TableHeader 
«/p» 
va 


## Create a string variable with all 
## the connection details. 
$connectionDetails = "Provider=sqloledb; " + 
"Data Source-$dataSource; " + 
"Initial Catalog=$database; " + 


"Integrated Security=SSPI;" 


## Connect to the data source, and 

## execute the T-SQL command. 

$connection = New-Object System.Data ~ 
.OleDb.OleDbConnection $connectionDetai ls 

$command = New-Object System.Data.OleDb ` 
-OleDbCommand $sqlCommand, $connection 

$connection.Open() 


Share your SQL Server code, 
comments, discoveries, and 
solutions to problems. Email 
your contributions to r2r@ 
sqlmag.com. Please include 
your full name and phone 
number. We edit submissions 
for style, grammar, and 
length. If we print your 
submission, you'll get $100. 
—Karen Bemowski, 
senior editor 


## Put the command's results into a DataSet 
## object, and close the connection. 
$dataAdapter = New-Object System.Data * 

.0leDb.OleDbDataAdapter $command 
$dataSet = New-Object System.Data.DataSet 
$dataAdapter.Fill($dataSet) 
$connection.Close() 


## Pipe the returned information into 
## ConvertTo-HTML, then pipe the HTML 
## page to the output file. 
$dataSet.Tables | 
Select-Object -Expand Rows | 
ConvertTo-HTML -head $a -body $body | 
Out-File $0utputFile 
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Figure I 


Sample results from the pub database 


Aaron Williams 


8 July 2010 


from the pubs sample database. As you can see, the 
HTML formatting code provides a few cosmetic 
touch-ups. 

As I mentioned previously, we use this PowerShell 
step for all our monitoring jobs. We simply modify the 
variables in callout A. Once the HTML file has been 
created, the job sends an email message that includes 


the link to the HTML file so DBAs can investigate 
any problems. 

If we need to add columns to or remove columns 
from a report, we update the T-SQL code. The T-SQL 
code is rolled into a stored procedure, so we simply use 
an ALTER PROCEDURE statement to update it. 

If you need to find a replacement for your 
sp makewebtask jobs, you might want to take a look 
at the ConvertTo-HTML cmdlet. PowerShell is a very 
powerful tool. If you're new to it, I recommend that 
you check out the “Hey, Scripting Guy!” columns listed 
on TechNet's "Scripting with Windows PowerShell” 
web page (technet.microsoft.com/en-us/scriptcenter/ 
dd742419.aspx). 

—Adam Moericke, lead DBA, Thomson Reuters 
InstantDoc ID 125232 


Search Through Code on 
Multiple Databases 


s à SQL Server programmer, there have 

been many occasions when I needed to 

search through code I’ve previously written 
to find a particular word or phrase. This isn't too 
difficult to do using SQL Server's built-in Catalog 
Views. For example, a T-SQL statement that searches 
for a particular word in the AdventureWorks sample 
database might look like that in Listing 2. The main 
problem with this approach is that it works only within 
the context of the current database. Often, I need to 
search all the databases on a SQL Server instance to 
find a function or stored procedure I had written. 

I needed a way to apply the code in Listing 2 to 
every database. The best way to do this is using an 
undocumented stored procedure in the master database 
named sp MSforeachdb. As its name implies, this 
stored procedure will execute a given T-SQL statement 


LISTING 2:Traditional T-SQL Statement That 
Looks for a Word in Code 


USE ADVENTUREWORKS ; 
DECLARE GSEARCH STRING VARCHAR(5Ø) 
SET GSEARCH STRING = 'CheckDate' 


SELECT 
sys.schemas.name AS [Schema Name], 
sys.objects.name AS [Object Name], 
type desc AS [Object Type], 
substring(definition,CHARINDEX(QSEARCH STRING, 
definition)-30, 69) 
AS [Text Context] 
FROM sys.objects 
JOIN sys.sql modules 
ON sys.objects.object id - 
JOIN sys.schemas 
ON sys.objects.schema id = sys.schemas.schema id 
WHERE CHARINDEX(GSEARCH STRING, sys.sql modules.definition) > Ø 


sys.sql modules.object id 


against every database on a SQL Server instance. The 
T-SQL statement is passed as a NVARCHAR(2000) 
parameter to sp MSforeachdb. 

The syntax for using sp MSforeachdb is 


EXEC sp MSforeachdb 'T-SQL code here' 


For example, suppose the current instance of SQL 
Server contains six databases: master, tempdb, model, 
msdb, AdventureWorksDW, and AdventureWorks. If 
you execute the code 


EXEC sp MSforeachdb 'SELECT getdate()' 


six separate queries will run and you'll receive the 
current date and time for each database. 

The sp MSforeachdb stored procedure also 
provides an easy way to display or use the current 
database name within your executable statement: Any 
question mark (?) you place within the parameter is 
automatically replaced by the name of the current 
database. For example, the command 


EXEC sp MSforeachdb 
"print ''The current db is ?''' 


results in the following being displayed: 


The current db is master 

The current db is tempdb 

The current db is model 

The current db is msdb 

The current db is AdventureWorksDW 
The current db is AdventureWorks 
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LISTING 3: SearchThroughAllCode.sql Object Type Text Context 


uspGetBillOt Materials, SOL STORFD PROCEDURE @StartProductiD [iet], ØCheckDate [datetime] AS REGI 
CREATE PROCEDURE [dbo] . [SearchThroughA11Code] wspGetWhereUsedProductlD SOL STORED PROCEDURE —@StartProductiD [int], @CheckDate [datetime] AS BEGI 
@searchStringIn varchar(255) AS 
BEGIN ; 
SET NOCOUNT ON; Figure 7 
A aa hee VARCHARC2000) Sample results from running the SearchThroughAllCode stored procedure 
SELECT ''?'' AS DATABASE NAME, : 
sys.schemas.name AS [Schema Name], stored procedure named SearchThroughAllObjects. 
Sys.objects.name AS [Object Name] , Using the same techniques as SearchThroughAll- 
type desc AS [Object Type], . 
substring(definition, Code, SearchThroughAllObjects searches for words 
PEE ed XXE, and phrases in tables, columns, synonyms, constraints, 
AS [Text Context] and linked servers as well as in stored procedures, 
FROM sys.objects views, and functions on each database on the SQL 
JOIN sys.sql modules . 
ON sys.objects.object id = Server instance. The syntax for SearchThrough- 
Sys.sql modules.object id AllObjects is 


JOIN sys.schemas 
ON sys.objects.schema id - 


MES EE. E EXECUTE [dbo].[SearchThroughAllObjects] 
definition)>9' 'SearchString' 
CREATE TABLE SEARCH RESULTS e 
( DATABASE NAME NVARCHAR(128), where SearchString is the word or phrase you want 
ee men to search for. SearchThroughAllObjects takes a 
[OBJECT TYPE] VARCHAR (2Ø), little longer to run than SearchThroughAllCode, 
TEXT CONTEXT VARCHAR(62) ) i 1 
INSERT INTO ÆSEARCH RESULTS but when you have to find something, it can be a 
EXEC sp MSforeachdb @SQL lifesaver. [SQL] 
—Ai illi 
SELECT * FROM £SEARCH RESULTS aron Williams, 
ORDER BY chief computer programmer, 
ES [SCHEMA NAME], [OBJECT NAME] Preferred Managing Agency 


InstantDoc ID 125247 


By modifying the search code that works within 
a database (e.g., code like that in Listing 2) to use the 
sp MSforeachdb stored procedure, you can create a 
stored procedure that searches through the code in all 
the databases. Listing 3 shows this stored procedure, 
which is named SearchThroughAllCode. As you 
can see in callout A, it takes advantage of the ques- 
tion mark so that each database on the SQL Server 
instance is searched. 

The syntax for the SearchThroughAllCode 
stored procedure is 


EXECUTE [dbo].[SearchThroughAllCode] 
'SearchString' 


where SearchString is the word or phrase you want 
to search for. So, for example, if you want to search 
for CheckDate, you'd use the command 


EXECUTE [dbo].[SearchThroughAT1Code] 
'CheckDate' 


Figure 2 shows sample results from running the 
stored procedure with this command. 

You can download the Search ThroughAllCode 
stored procedure by going to www.sqlmag.com, AL Se: € 
entering 125247 in the InstantDoc ID text box, DIS 1 , SSH, Amazon S3, OpenPGP, USPS, 
clicking Go, then clicking the Download the Code B2, C nix, Salesforce, ODBC, MySQL Scripts 
Here button. This download also includes another om | 919-249-7421 | sales@cozyroc.com 
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QUESTIONS 
ANSWERED 


Why Is a Rolled-Back Transaction 
Causing My Differential Backup to 
Be Large? 


Paul S. Randal 


Kimberly L. Tripp 


Paul $. Randal (paul @ SQLskills.com) 

and Kimberly L. Tripp (kimberly@ 
SQLskills.com) are a husband-and-wife 
team who own and run SQLskills.com, a 
world-renowned SQL Server consulting and 
training company. They're both SQL Server 
MVPs and Microsoft Regional Directors, 
with more than 30 years of combined 
SQL Server experience. 


D Editors Note 


Check out Kimberly and 
Paul's blog, Kimberly & 
Paul: SQL Server Questions 
Answered, on the SQL Mag 
website at www.sqlmag.com. 


] 
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differential backups. When I update a large por- 

tion of a table (say, 10GB), the next differential 
backup is about 10GB. When I perform the same 
update and roll back the transaction, the next differ- 
ential backup is still 10GB. Why isn't the differential 
backup empty in the second case? 


Å ^m confused by some behavior I’ve noticed with 


You're correct that the behavior doesn't seem 
intuitive at first—nothing changed in the data, 
right? However, SQL Server is absolutely doing the 
right thing under the covers. The explanation of this 
behavior has two parts. 

First, I'll explain how SQL Server knows what to 
include in a differential backup. The behavior is the 
same for file, file groups, and database differential 
backups (Tl just call them differential backups for 
simplicity). A differential backup includes every- 
thing that has changed since the last full backup, 
with the minimum unit of change being an extent 
(i.e., a 64K B chunk of a data file comprised of 8KB 
x 8KB data file pages). 

When a change is made to a data file page, the 
extent it's a part of is marked as changed in a special 
bitmap called the differential bitmap or differential 
change map. There's a differential bitmap tracking 
each 4GB portion of each data file. A differential 
backup scans the differential bitmaps for the por- 
tion of the database being backed up and includes 
all extents marked as changed. You can read more 
about the differential bitmap and other specialized 
allocation bitmaps in my blog post "Inside The 
Storage Engine: GAM, SGAM, PFS and other allo- 
cation maps” (www.sqlskills.;com/BLOGS/PAUL/ 
post/Inside-The-Storage-Engine-GAM-SGAM- 
PFS-and-other-allocation-maps.aspx). 

A full backup is the only operation that clears the 
differential bitmaps, so subsequent differential backups 
without intervening full backups can become larger 
and larger as more of the database changes. You can see 
how much of a database has changed since the last full 
backup using the script in my blog post “New script: 
How much of the database has changed since the last 
full backup?" (www.sglskills.com/BLOGS/PAUL/post/ 
New-script-How-much-of-the-database-has-changed- 
since-the-last-full-backup.aspx). Some people use 


this script in production to help automatically decide 
when to take a full backup instead of another differ- 
ential backup, and it can also give you a measure of 
the churn rate of the database. 

The second part of the explanation gets to the crux 
of your question—why doesnt rolling back a transac- 
tion clear the differential bitmap bits that the transaction 
caused to be set in the first place? Well, it can't. 

Simply put, every change made to a SQL Server 
database must generate a transaction log record 
describing the change. A complex operation might 
generate many transaction log records, each describing 
a change to a data file page. Each data file page has a 
field in the page header (1.e., the first 96 bytes of the 
page that contain metadata about the page and its 
contents) that tracks the last transaction log record 
that affected that page. This field 1s used for crash- 
recovery purposes, which is beyond the scope of this 
article, and each transaction log record tracks which 
data file page it affected. You can read more about 
page header fields and their meanings in my blog 
post "Inside the Storage Engine: Anatomy of a page" 
(www.sqlskills.com/BLOGS/PAU L/post/Inside-the- 
Storage-Engine-Anatomy-of-a-page.aspx). 

When a transaction rolls back, all of the changes 
it made in the database essentially have to be 
reversed. Performing the reverse of an INSERT isa 
DELETE, the reverse of a DELETE is an INSERT, 
and so on. This process is driven from the transac- 
tion log records the transaction generated—they’re 
undone in the opposite order from which they were 
generated by the original transaction. 

The main point here is that rolling back a 
transaction involves performing more changes to 
the database and generating more transaction log 
records. This means the data file pages involved 
are changed yet again, even though the change is 
undoing the effects of the first change. 

As far as the logical operation of backups 1s 
concerned, it doesn't matter whether the pages in a 
data file extent were changed by a transaction that 
was subsequently rolled back—they were changed, 
so they have to be backed up by the next differen- 
tial backup. That's why a rolled-back transaction 
still results in data being included in a differential 
backup. 
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UESTIONS 
NSWERED 


Partial Database Availability 
for Targeted Restores 


have a large database that Pd like to restore on a 

development system pretty regularly, but I don't 

want to restore all the data. Is there a way to 
perform a partial restore without creating multiple 
databases? 


The easiest way to do so is to split the database 
into multiple file groups and restore only some of 
the file groups. This procedure uses an Enterprise 
Edition-only feature called partial database avail- 
ability (available in SQL Server 2005 and later), 
which allows a database to be online as long as 
the primary file group is online (obviously, the 
file groups that are offline aren't accessible). If 
your development system runs on the much-less 
expensive Developer Edition, you'll still be able to 
use partial database availability because Developer 
Edition is basically Enterprise Edition that you just 
can't use in production. 

In addition to enabling the scenario you describe, 
partial database availability is a fantastic way to 
reduce downtime during disaster recovery using 
backups, because the most critical portion of the 
data can be brought online first, and other portions 
are restored using online piecemeal restore while the 
main workload is running. 

The two easiest ways to split the data into 
multiple file groups are to place tables in separate 
file groups manually or to partition individual tables 
using table and index partitioning. 

With manual partitioning, you can isolate one 
or more tables (and their indexes) into each file 
group. For example, if you had a parts sales data- 
base, you might decide to create a sales table for 
each state, plus a parts table, and have each table 
in its own file group. In your scenario, you might 
pick the smallest sales file group and the parts file 
group to restore to the development system. In a 
disaster recovery scenario, you might choose to 
restore the sales file group for the state with the 
largest population first (to minimize the down- 
time for the portion of the database handling the 
most sales), bring the database online, and then 
restore the file groups for the remaining states in 
sequence. 

Using SQL Servers table/index partitioning 
feature, you can perform horizontal partitioning 
on a table (and indexes) and have each portion of 
the table in a separate file group. For instance, in a 
sales database that contains the last few years' data, 
you might partition by date and have a separate file 
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group for each year’s data. In your scenario, you 
would likely restore just the current year's data on 
the development system to test the OLTP system. In 
a disaster recovery scenario, you'd restore the current 
year's data first to be able to bring the OLTP work- 
load online, and restore the historical data later. 

Once you've split the database into multiple 
file groups, you can restore a subset of them by 
performing a partial restore. First, you'll create a 
database with three file groups, then you'll put a 
table on each of the non-primary file groups using 
the code in Listing 1. 

Now on a different system you can restore just 
the primary file group and the first file group. The 
key 1s to restore the primary file group first using 
WITH PARTIAL to let SQL Server know you aren't 
restoring the entire database, as shown in Listing 2. 

Next, you can restore any differential and/or 
transaction log backups to get the restored file 
groups to the desired point in time, and then bring 
the database online using the following code: 


RESTORE DATABASE PaulsDB WITH 
RECOVERY ; 
GO 


If I query the database using the code in Listing 3, 
it will show me what's online. 


LISTING |: Code to Create a Database with Multiple 
File Groups 


CREATE DATABASE PaulsDB; 
GO 


ALTER DATABASE PaulsDB ADD FILEGROUP TestFileGroup1; 
ALTER DATABASE PaulsDB ADD FILEGROUP TestFileGroup2; 
GO 


ALTER DATABASE PaulsDB ADD FILE ( 
NAME = TestFilel, 
FILENAME = 'C:\SQLskills\TestFilel.ndf', 
SIZE = 5MB) 

TO FILEGROUP TestFileGroup1; 

ALTER DATABASE PaulsDB ADD FILE ( 
NAME - TestFile2, 
FILENAME = 'C:\SQLskills\TestFile2.ndf', 
SIZE = 5MB) 

TO FILEGROUP TestFileGroup2; 

GO 


CREATE TABLE PaulsDB..t1 (c1 INT) ON TestFileGroupl; 
CREATE TABLE PaulsDB..t2 (c1 INT) ON TestFileGroup2; 
GO 

INSERT INTO PaulsDB..t1 VALUES (1); 

INSERT INTO PaulsDB..t2 VALUES (2); 

GO 


BACKUP DATABASE PaulsDB TO DISK = 'c:\SQLskills\ PaulsDB.bck'; 
GO 


July 2010 


QUESTIONS 
ANSWERED 


LISTING 2: Restoring the Primary 


File Group Using WITH PARTIAL 


RESTORE DATABASE PaulsDB 
FILEGROUP = 'primary' 
FROM DISK = 'c:NSQLskillsN PaulsDB.bck" 
WITH PARTIAL, NORECOVERY; 
GO 


RESTORE DATABASE PaulsDB 
FILEGROUP = 'TestFileGroupl' 
FROM DISK = 'c:NSQLskillsN PaulsDB.bck" 
WITH NORECOVERY; 
GO 


LISTING 3: Code Used to Determine 
Which File Groups Are Online 


SELECT [name], [state desc] FROM PaulsDB 
.Sys.database files; 


GO 

name state desc 
PaulsDB ONLINE 

PaulsDB log ONLINE 
TestFilel ONLINE 

TestFile2 RECOVERY PENDING 


== Audit Database changes, 
Read the SQL Transaction Log 


ApexSQL Log 2008 
The Ultimate Log Reading, Auditing Tool 

for SQL Server 

mund SQL Server 2008 and 64-bit support 

m Integrates with Database backups for improved audit trails 
W Analyze historical activity of each transaction 

Y Rollback or reconstruct operations 

W Export transactions to XML, BULK SQL, CSV, etc. 


W Access to programmable API for automation 


W Command line interface included as standard feature A 


Now, of course, I can only use the portions of 
the database that I've restored. If I try to access 
anything in an offline file group, lll get an error 
similar to the following: 


SELECT * FROM PaulsDB..t2; 

GO 

Msg 8653, Level 16, State 1, Line 1 

The query processor is unable to 
produce a plan for the table or view 
't2' because the table resides in a 
filegroup which is not online. 


For more information about using the native 
partitioning feature, I recommend reading two 
excellent white papers: “Partitioned Tables and 
Indexes in SQL Server 2005" (msdn.microsoft.com/ 
en-us/library/ms345146.aspx), which was actually 
written by Kimberly, and “Partitioned Table and 
Index Strategies Using SQL Server 2008" (msdn 
.microsoft.com/en-us/library/dd578580.aspx), which 
uses a different example scenario. E 


L pec cc oe = ISOLE] 
(CER O BINE 
For more information BEST 


Y Read and analyze SQL Server transaction logs or to download a free trial version go to: 


ApexSQL  www.apexsgl.com 
software or phone 866-665-5500 
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Left-Brain.com is the newly launched online superstore stocked with 
educational, training, and career-development materials focused on 
meeting the needs of SOL Server professionals like you. 


> SC Serverzox System Views Featured Product: 
== SQL Server 2008 System Views Poster 


Face the migration learning curve head on with the SQL Server 
2008 System Views poster. An updated full-color print diagram of 
catalog views, dynamic management views, tables, and objects for 
SQL Server 2008 (including relationship types and object scope), 
this poster is a must-have for every SOL DBA migrating to or al- 
ready working with SOL Server 2008. 


Order your full-size, print copy 
today for only $14.95*! 


*Plus shipping and applicable tax. 
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Rodney Landrum 


(rlandrum 13 @ cox.net) oversees a large 


SQL Server infrastructure in Pensacola, 


Florida. He regularly writes about SQL Server 


technologies for SQL Server Magazine and 
Simple-Talk.com. He's a SQL Server MVP and 
the author of three books on SQL Server 
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Reporting Services. 


ORE on the WEB 


in 2007. Since then, I've received many email 
messages with feedback and requests, which has 
left me both honored and chagrined. Im honored 
that many DBAs have incorporated this solution into 
their own environments, often tweaking, tuning, and 
ultimately making it their own. I’m chagrined because 
I’ve been remiss in updating the DBA Repository 
lately. ve now remedied that situation by making the 
following enhancements: 
e Resiliency for offline servers 
e SQL Server 2008 support 
e Performance improvements through the introduc- 
tion of parallel processing 


| shared my DBA Repository solution back 


In case you're unfamiliar with the DBA Reposi- 
tory, at its heart lies a SQL Server Integration Ser- 
vices (SSIS) package that gathers information DBAs 
can use to assess and report on the SQL 
Server machines they manage. For example, 
information about server names, SQL Server 


Download the code at 


InstantDoc ID 125240. 


editions and service pack levels, database 
names and sizes, user and login permissions, 
and SQL Server Agent jobs is gathered. This 
data is stored in a central database named DBA_Rep. 
You can find more information about the original 
solution and its initial updates in the articles listed in 
the Learning Path box. 


Resiliency for Offline Servers 

When I first built the DBA Repository, I was under 
the gun to get a solution working quickly. My main 
goal was to use the repository as a documentation 
source for myself and the other DBAs I worked with. 


Although I was concerned with error handling within 
the SSIS package, I didn’t spend as much time as I 
could have on it. When I executed the package with 
a scheduled SQL Server Agent job, I soon found out 
that a server was offline when the job ran, raising 
an error. Because the package’s processing was syn- 
chronous (i.e., completed one step at a time), all the 
information that would have been gathered after the 
error occurred was lost. I had to resolve this prob- 
lem before a successful run of the repository package 
could be performed. 

There's a property named MaximumErrorCount 
whose value you can modify to accommodate a higher 
error threshold for an SSIS package. By default, this 
property's value is set to I, which means that one error 
is the maximum number allowed. When this thresh- 
old is reached, all processing stops at that point. As 
a short-term solution, I changed this property's value 
to a higher number. 

Most objects in SSIS packages include the Maxi- 
mumErrorCount property. As the package grew larger, 
I found myself adjusting this property for each object, 
which quickly became tiresome. At the time, I also con- 
figured a simple SMTP object in the Event Handler 
that would email me the name of the offline server, but 
this didn't prevent the entire package from failing. 

I’ve now devised a solution that provides resil- 
iency for offline servers. The solution is fairly simple 
but effective. Instead of expecting that all the servers 
will be available, I added a ForEach Loop container 
that attempts a connection with each server before 
the data collection begins. 

The list of target servers is created with a T-SQL query 
that pulls data from a table named ServerList. SSIS table. 
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In the original solution, this table contained five fields: 
one varchar field to store the server name and four 
smallint fields (Connect, Version, DMZ, and Loca- 
tionID) to control the flow of processing within the 
SSIS package. To gain the resiliency I needed, I added 
another field named Online to this table. The Online 
field is used to check each server in the server list and 
report whether it's online or offline to the package and 
ultimately to you. It also prevents the package from 
accumulating connection errors. 

Figure 1 shows the Loop for Server Checks section I 
added to the SSIS package to make an initial spin through 
all of the non-demilitarized zone (non-DMZ) servers 
to make sure they're online. The logic in this section is 
simple. The Check Servers Execute SQL task executes 
a single statement against all the servers (one server at a 
time) in the populated global variable SRV. Conn. (Two 
variables—SRV Conn and SQL RS—are passed to 
the ForEach Loop container and populated via Execute 
SQL tasks. The process is described in detail in *SQL 
Server Integration Services,” May 2007, InstantDoc ID 
95385.) If the server is available and returns a result, the 
Check Servers task succeeds and passes the value to the 
Update Server List Success task. If the server is offline, 
an error occurs and the workflow tells the Check Servers 
task to execute the Update Server List Failure task. 

Both the Update Server List Success and Update 
Server List Failure tasks execute a T-SQL UPDATE 
statement to update the ServerList SSIS table. The 
UPDATE statement sets the Online field to either 
a 1 for success or 0 for failure. As Figure 2 shows, 
the UPDATE statement uses the value of the current 
SRV Conn variable by means of parameter mapping 
in the Update Server List Success or Update Server 
List Failure task. With the mapping established, all 
that is required is to use the variable via a question 
mark (?) in the UPDATE statement: 


UPDATE ServerList SSIS 
SET Online = 1 
WHERE Server = ? 


After all the servers have been queried, the Loop 
for Server Checks section completes execution and 
passes the workflow simultaneously to two additional 
Execute SQL tasks: Send Mail of Offline Servers and 
Load All Online Servers to Variable. As the name 
suggests, the Send Mail of Offline Servers task que- 
ries the ServerList SSIS table for the freshly updated 
list of unavailable servers and sends a notification 
email to the specified DBA using the system stored 
procedure msdb..sp send dbmail. In the previous 
version of the DBA Repository SSIS package, the 
DBA received a separate email notification for each 
offline server. In the updated SSIS package, the DBA 
receives one email message for each server group 
(DMZ and non-DMZ) when there are failures. 
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DBA REPOSITORY D i OR 


LEARNING PATH 


For background information about the DBA Repository, see 


“Use SSRS and SSIS to Create a DBA Repository,” 
InstantDoc ID 97840 


“SQL Server Reporting Services,” 
InstantDoc ID 95745 


“SQL Server Integration Services,” 
InstantDoc ID 95385 


"Sglemd," InstantDoc ID 95387 


Load al 


dg] Server 


5i All Servers Query for 
L7 SQL RS Variable 


g9 Loop for Server Checks 


lg Check Servers lur 


m Update Server 


x| List Success 


A, Update Server 
lg List Failure 


rs, Send Mail of Offline 
(E Servers 


Figure I 


New process for checking whether servers are online 


IE; Execute SQL Task Editor 


Configure the properties required to run SQL statements and stored procedures using the selected 


n connection. 


General | Variable Name Direction Data Type Parameter... Parameter. 
Parameter Mapping VARCHAR 0 50 

Result Set 
Expressions 


> 


Add Remove 


Figure 2 


Parameter mapping for the Update Server List Success and Update Server List 
Failure tasks 
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DBA REPOSITORY 


Listing 1 shows the T-SQL code that's executed 
for the Send Mail of Offline Servers task. Notice that 
the message will go out only when there is one or 
more offline servers discovered. The logic to send a 
message only when servers are offline is accomplished 
by an IF (BEGIN...END) construct around the 
sp send, dbmail stored procedure. 


SQL Server 2008 Support 

The SSIS package needs information about the SOL 
Server version that's installed on each machine being 
polled because certain sections within the package are 
version dependent. For example, the job scheduling 
section needs to know which SQL Server version is 
running because, in msdb, the schema for storing the 
job scheduling information is different in SQL Server 
2000 than in SQL Server 2005 and SQL Server 2008. 
In addition, with the introduction of parallel process- 
ingin the DBA Repository solution, the version infor- 
mation is even more important because some data is 
gathered simultaneously for all servers, regardless of 


LISTING I: Code that Sends a Message About Offline 


Servers 


IF (SELECT count(*) FROM dba rep..ServerList SSIS WHERE Online - Ø) > Ø 


BEGIN 


EXEC msdb..sp send dbmail 


Gprofile name = 


'Notifications', 


@recipients = 'YouremailaddressGyourmaildomain.com, 
@subject = 'DBA Repository Offline Servers' , 
@query = 'SET NOCOUNT ON; 

PRINT ''OFFLINE SERVERS @ '' ; 


SELECT GETDATEO ; 
PIN 


SELECT Server FROM ServerList SSIS WHERE online = £' , 
Gexecute query database =  'DBA Rep' , 
Qquery result width = 40, 
Qquery result header - 6 


END 


il, Load All Online Servers 
LE V] to variable 


IE; Execute SQL Task Editor 


les) connection. 


fe RePopulate All Servers 


version, while other data is selectively gathered based 
on the version. 

Previously, the logic for distinguishing SQL 
Server versions was controlled by a rudimentary 
string parsing routine. The string parsing routine 
grabbed the first character of the Product_Version 
field in the SQL_Servers table. (This DBA_Rep 
database table contains the information gathered 
from the online servers.) The Product_Version 
field was character-based, holding values such as 
9.00.4035.00. The string parsing routine then com- 
pared that character to a value—8 for SQL Server 
2000 (which is version 8.x) and 9 for SQL Server 
2005 (which is version 9.x) —by making a numeric 
comparison such as 


WHERE value > 8 


This string parsing routine doesn’t work with SQL 
Server 2008 because its version number is 10.x. 
Grabbing a single character from a value such as 
10.0.1600.22 would result in comparing the character 
of 1 instead of 10. 

Admittedly, this was an oversight in my original 
solution. Fixing this problem so that the SSIS package 
would work on SQL Server 2008 machines involved 
modifying the SQL Servers table and updating the 
SSIS package. 

Modifying the SQL Servers table. I needed a new 
field for driving the package flow. SQL Servers is the 
first table to be populated in the package and it held 
the original Product Version field, so it made sense 
to add the new field to that table. The new field, 
Product Version Int, holds the integer value of the 
SQL Server version and has a data type of smallint. 


Configure the properties required to run SQL statements and stored procedures using the selected 


Update Serverlist SSIS Product Yer: 


Execute SQL Task 


General El General 
Parameter Mapping Name 
Result Set Description 


Expressions El Options 


a Load Server Info TimeOut 0 
CodePage 1252 
q9 Load Servers 


El Result Set 
ResultSet None 
E SQL Statement 
ConnectionType OLE DB 
Connection QASRV.DBA REP 
SQLSourceType Direct input. 


men! update serverlist_SSIS set version = SQL Servers. Product, Version Int 
From ServerList_SSIS Inner Join SQL Servers on 
ServerList SSIS.Server = SQL Servers.Server 


i | v] Product Version Ol 


j B Update Serverlist. SSIS 


StoredProcedure 


! BypassPrepare 
1 Name 


Specifies the name of the task. 


Server version is 
gathered in the 
SQL Servers table 
using ServerList SSIS is 

@@microsoftversion next updated with — 
/ power(2, 24) the accurate 
version from 

SQL Servers table 


Parse Query 


Build Query... l 


[ OK | Cancel 


Figure 3 


New process for obtaining the SQL Server version 
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ROCKING APPFABRIC ACCESS CONTROL: 
PRACTICAL SCENARIOS, KILLER CODE AND 
WICKED TOOLS 

MICHELE LEROUX BUSTAMANTE 


A SIMPLE UI SHELL FOR XAML 
APPLICATIONS ROCKY LHOTKA 


BUILDING A WPF/SILVERLIGHT UI IN VISUAL 
STUDIO 2010 ROCKY LHOTKA 


SECURING WPF CLIENT APPLICATIONS 
BRIAN NOYES 


BUILD N-TIER SILVERLIGHT DATA 
APPLICATIONS EASILY WITH WCF 
RIA SERVICES BRIAN NOYES 


BUILD COMPOSITE WPF AND SILVERLIGHT 
APPLICATIONS WITH PRISM 
BRIAN NOYES 


.NET ROCKS! LIVE 
RICHARD CAMPBELL AND 
CARL FRANKLIN 


DIGGING INTO THE WINDOWS PHONE 7 
ARCHITECTURE 
NICKOLAS LANDRY 


BUILDING INTEGRATED MOBILE 
APPLICATIONS WITH WINDOWS PHONE 7 
HARDWARE & SERVICES 

NICKOLAS LANDRY 


MOBILE APPLICATION DEVELOPMENT 
(MAD) MONEY: $ELLING $OFTWARE 
NICKOLAS LANDRY 


DEVICE & SERVER: NETWORKING & 
NOTIFICATIONS IN WINDOWS PHONE 7 
APPLICATIONS NICKOLAS LANDRY 


LANGUAGE FEATURES IN 4.0 TBD 


WINDOWS AZURE / CLOUD COMPUTING 
TBD 


CLR 4.0 INNOVATION TBD 
NEW INTEROP FEATURES TBD 


NEW FEATURES IN VISUAL STUDIO 2010 
TBD 


(PRE-CONFERENCE WORKSHOP) 
MONDAY, NOVEMBER 1, 2010 9AM - 4PM 
EVERY CLASS AS A SERVICE — 

WCF AS THE NEW .NET 

JUVAL LOWY 


(POST-CONFERENCE WORKSHOP) 

FRIDAY, NOVEMBER 5, 2010 9AM - 4PM 
FEDERATED IDENTITY ESSENTIALS 
MICHELE LEROUX BUSTAMANTE 


Also in-depth sessions delivered 
by Microsoft's Visual Studio 
team to be posted on our 
website July 1st. 
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Microsoft® 


ASPnet 


CONNECTIONS 


Come to Microsoft ASP.NET 
Connections in Las Vegas in 
November to hear from the 
best and brightest in the indus- 
try as you increase your 
ASP.NET development skills! 
Discover new features in Web 
Forms and ASP.NET MVC 2 that 
will make website development 
faster, perform better and 
more secure. Have you won- 
dered if you should run your 
websites in Windows Azure? 
Get the answer at ASPNET 
Connections. Our speakers 
include ASPNET co-creator and 
Vice President at Microsoft, 
Scott Guthrie, as well as 
Microsoft Program Managers 
and leading ASP.NET gurus 
outside of Microsoft. 


Microsoft® 
Silverlight’ 


Looking for real-world guid- 
ance that you can apply to 
your line of business, Windows 
Phone 7 or consumer-facing 
Silverlight applications? The 
DevConnections Silverlight 
track is packed with exciting 
and practical content that you 
can use to enhance your 
design skills, integrate data 
from distributed sources, learn 
best practices, build architec- 
turally sound applications and 
more. All sessions are present- 
ed by top-notch speakers who 
use Silverlight, Expression 
Blend and Visual Studio 2010 
to build client applications in 
the real-world, so you'll walk 
away with expert guidance 
that will help jumpstart your 
development projects. 


GUIDING PRINCIPLES FOR ASP.NET 
MVC APPLICATIONS. SCOTT ALLEN 


LEARNING TDD WITH ASP.NET MVC 
SCOTT ALLEN 


EVERYTHING NEW FOR ASP.NET MVC 
SCOTT ALLEN 


CASCADING STYLES SHEETS (CSS) FOR 
ASP.NET DEVELOPERS 
ROBERT BOEDIGHEIMER 


MAXIMIZE REUSE IN YOUR ASP.NET 
WEB SITES ROBERT BOEDIGHEIMER 


THINGS EVERY ASP.NET DEVELOPER 
SHOULD KNOW ROBERT BOEDIGHEIMER 


BUILDING HTML HELPERS FOR ASP.NET MVC 
DINO ESPOSITO 


SHOULD YOUR APPLICATION RUN IN AZURE? 
STEVE EVANS 


NETWORKING FOR DEVELOPERS 
STEVE EVANS 


IIS 7.5 WEB FARMS FOR DEVELOPERS 
STEVE EVANS 


BUILDING N-TIER ASP.NET WEB FORMS WITH 
ENTITY FRAMEWORK JULIE LERMAN 


ASP.NET MVC FOR WEB FORMS 
PROGRAMMERS PAUL LITWIN 


BRING CHARTING TO YOUR ASP.NET SITES 
WITH THE CHART CONTROL 
PAUL LITWIN 


THE ZEN OF ASP.NET AND MVC 
JAVIER LOZANO 


OH ASP.NET MVC. HOW EXTENSIBLE 
ART THOU? JAVIER LOZANO 


OPEN SOURCE TOOLS EVERY .NET DEVELOPER 
SHOULD USE JAVIER LOZANO 


CREATE DYNAMIC THEMES FOR ASP.NET 
WEB FORMS J. MICHAEL PALERMO 


JQUERY FOR THE ASP.NET WEB FORMS GUY 
J. MICHAEL PALERMO 


GETTING STARTED WITH ASP.NET WEB 
FORMS MVP J. MICHAEL PALERMO 


PROGRAMMING TWITTER AL PASCUAL 


WEB FORMS ARE FOR THE ENTERPRISE 
AL PASCUAL 


INTRODUCTION TO WINDOWS AZURE 
WALLY MCCLURE 


WHAT'S NEW IN ASP.NET 4 FOR 
WEB FORMS? WALLY MCCLURE 


AJAX WITH THE UPDATEPANEL, WEB FORMS, 
AND THE AJAX CONTROL TOOLKIT 
WALLY MCCLURE 
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HIGH-SPEED AJAX APPLICATIONS 
CHRISTIAN WENZ 


SEARCH ENGINE OPTIMIZATION WITH 
ASP.NET CHRISTIAN WENZ 


WEB APPLICATION SECURITY TRENDS 
CHRISTIAN WENZ 


(PRE-CONFERENCE WORKSHOP) 

MONDAY, NOVEMBER 1, 2010 9AM - 4PM 
BUILDING APPLICATIONS WITH ASP.NET MVC 
SCOTT ALLEN 


(POST-CONFERENCE WORKSHOP) 

FRIDAY, NOVEMBER 5, 2010 9AM - 4PM 
BUILDING AJAX-ENABLED APPLICATIONS 
WITH JQUERY DAN WAHLIN 


SILVERLIGHT SESSIONS 


SILVERLIGHT DEVELOPER'S GUIDE TO ENTITY 
FRAMEWORK WARD BELL 


SECURING SILVERLIGHT APPS: WHAT YOU 
DON'T KNOW CAN HURT YOU WARD BELL 


AUTOMATED TESTING IN SILVERLIGHT 
WARD BELL 


MEF AND RX-BASED MVVM 
KATHLEEN DOLLARD 


FROM 0 TO STYLIN' IN 60 MINUTES — ADDING 
STYLE TO SILVERLIGHT 4 APPLICATIONS 
EVAN HUTNICK 


BUILDING BI DASHBOARDS WITH 
SILVERLIGHT 4 AND WCF RIA SERVICES 
EVAN HUTNICK 


GETTING STARTED WITH AND CUSTOMIZING 
YOUR SILVERLIGHT NAVIGATION APPLICATION 
EVAN HUTNICK 


BREAKING DOWN WALLS — THE STORY OF 
GETTING DESIGNERS AND DEVELOPERS 
WORKING TOGETHER IN AN AGENCY 
ENVIRONMENT DAVID KELLEY 


GOING FROM SILVERLIGHT TO WINDOWS 
PHONE 7 APPLICATION DEVELOPMENT 
DAVID KELLEY 


EXPRESSION BLEND IS NOT ONLY FOR 
DESIGNERS JOEL NEUBECK 


(PRE-CONFERENCE WORKSHOP) 

MONDAY, NOVEMBER 1, 2010 9AM - 4PM 
SILVERLIGHT 4 DEVELOPMENT WORKSHOP 
DAN WAHLIN 


Also in-depth sessions delivered 
by Microsoft's ASP.NET & 
Silverlight teams to be posted on 
our website July 1st. 


SQL Server 


CONNECTIONS 


Celebrate over 10 years of 
SQL Server Connections as 
we explore the new features 
of SQL Server 2008 and SQL 
Server 2008 R2 but focus 

on showing you how to get 
the most out of existing 

SQL Server installations 

as you face ever-increasing 
demands for performance 
and availability. Our dynamic 
mix of industry experts 

will show you how to avoid 
common problems and how 
to mitigate problems you 
already have. Sessions will 
emphasize real-world lessons 
learned and best practices 
for security, consolidation, 
defensive programming, 
database deployment, 
Reporting Services, 
PowerPivot, and much more. 
We're also partnering with 
SQLServerCentral.com for 
an extra track of sessions on 
practical, real-world skills for 
both DBAs and developers 
with dynamic sessions from 
some of the most acclaimed 
speakers in the SQL Server 
community. 


SQL * 


ServerCentral.co: 


See website for 
SQLServerCentral.com 
session track. 


SPR302: PLUMBING THE DEPTHS OF SQL 
SERVER / POWERSHELL INTEGRATION 
BOB BEAUCHEMIN 


SPR201: DATABASE BEST PRACTICES FOR 
THE INVOLUNTARY DBA 
PAUL S. RANDAL & KIMBERLY L. TRIPP 


SDB305: DBA MYTHBUSTERS 
PAUL S. RANDAL 


SDB303: VLDB: RECOVERING FROM 
ISOLATED DISASTERS 
KIMBERLY L. TRIPP 


SDB306: INDEX FRAGMENTATION: 
THE HIDDEN MENACE 
PAUL S. RANDAL 


SDB407: UNDOCUMENTED TOOLS AND 
TRACE FLAGS PAUL S. RANDAL 


SPS301: INDEXING STRATEGIES AND ANALYSIS 
PAUL S. RANDAL & KIMBERLY L. TRIPP 


SDV308: PROGRAMMING FILESTREAMS AND 
RBS IN SQL SERVER 2008 R2 
BOB BEAUCHEMIN 


SDV309: BUILDING, DEPLOYING, AND 
MONITORING STREAMINSIGHT 
APPLICATIONS BOB BEAUCHEMIN 


SDV301: QUERY TUNING TIPS 
ITZIK BEN-GAN 


SDV302: AGGREGATING DATA IN SQL SERVER 
2008 USING GROUPING SETS 
ITZIK BEN-GAN 


SDV304: BEST PRACTICES FOR WORKING WITH 
DATE AND TIME DATA IN SQL SERVER 2008 
ITZIK BEN-GAN 


SDV303: USING THE SQL SERVER SERVICE 
BROKER IN THE REAL WORLD 
DENNY CHERRY 


SDV207: MITIGATING ADHOC QUERY ISSUES 
IN SQL SERVER 2008 RONALD YENKO 


SDB410: AUDITING AND COMPLIANCE 
FEATURES IN SQL SERVER 2008 
BOB BEAUCHEMIN 


SDB304: STORAGE FOR THE DBA 
DENNY CHERRY 


SDB309: HOW TO DECIDE IF YOUR DATABASE 
IS A GOOD CANDIDATE FOR VIRTUALIZATION 
DENNY CHERRY 


SDB301: CLUSTERING IN SQL SERVER 
RONALD YENKO 


SDB302: DATABASE MIRRORING: EVEN MORE 
ROBUST IN SQL SERVER 2008 
RONALD YENKO 


SBI204: CREATING REPORT SUBSCRIPTIONS 
IN MICROSOFT SQL SERVER 2008 REPORTING 
SERVICES PAUL LITWIN 


SBI305: PROGRAMMING SQL SERVER 2008 
REPORTING SERVICES PAUL LITWIN 


SBI201: DATA WAREHOUSE ETL WITH 
INTEGRATION SERVICES 
TODD MCDERMID 


SBI302: IN-DEPTH WITH THE SSIS 
SCRIPT TASK TODD MCDERMID 


SBI406: IN-DEPTH WITH THE SSIS SCRIPT 
COMPONENT TODD MCDERMID 


SBI203: POWERPIVOT: DELIVERING 
SELF-SERVE BI CRAIG UTLEY 


SBI307: DO IT RIGHT: BEST PRACTICES FOR 
ANALYSIS SERVICES CRAIG UTLEY 


SBI408: ADD IT UP: ANALYSIS SERVICES 
AGGREGATIONS CRAIG UTLEY 


SDV305: GUIDS: USE, ABUSE, AND HOW TO 
MOVE FORWARD KIMBERLY L. TRIPP 


SDV306: INDEX INTERNALS: WHAT YOU REALLY 
NEED TO KNOW! KIMBERLY L. TRIPP 


TOP-10 SQL SUPPORT ISSUES AND HOW TO 
SOLVE THEM TBD 


SUCCESSFUL SHARDING ARCHITECTURES 
TBD 


FAST-TRACK DATA WAREHOUSE TBD 


SQL SERVER R2 EDITIONS COMPARISON - 
HOW TO CHOOSE WHAT YOU NEED TBD 


(PRE-CONFERENCE WORKSHOP) 

MONDAY, NOVEMBER 1, 2010 9AM - 4PM 
DATABASE BEST PRACTICES FOR THE 
INVOLUNTARY DBA 

PAUL S. RANDAL AND KIMBERLY L. TRIPP 


(PRE-CONFERENCE WORKSHOP) 

MONDAY, NOVEMBER 1, 2010 9AM - 4PM 

DAY OF SCRIPTING: PLUMBING THE DEPTHS 
OF SQL SERVER / POWERSHELL INTEGRATION 
PRESENTED BY BOB BEAUCHEMIN 


(POST-CONFERENCE WORKSHOP) 

FRIDAY, NOVEMBER 5, 2010 9AM - 4PM 

SPS301: TUNING FOR HIGH PERFORMANCE 
PAUL S. RANDAL AND KIMBERLY L. TRIPP 


Also in-depth sessions delivered 
by Microsoft's SQL Server team to 
be posted on our website July 1st. 
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SharePoint 


CONNECTIONS 


Leading SharePoint experts 
from Microsoft and from the 
field have teamed up to bring 
to you the knowledge you need 
to succeed with SharePoint 
2010. 


IT PROS! Come hear 

Dan Holme, Michael Noel 

and others lay out the best 
practices for installing, upgrad- 
ing, configuring, securing, and 
managing SharePoint 2010. 

Go beyond the hype and dive 
deep into what it takes to 
successfully deploy SharePoint 
2010 in the real world. 


DEVELOPERS! Come hear 
Andrew Connell, Ted Pattison, 
Scot Hiller and others provide 
guidance on how to best cus- 
tomize and extend your 
SharePoint 2010 investments 
using the new data access 
methods on the server (LINQ) 
and off the server (client object 
model) leveraging Silverlight, 
working with data that does 
not live within SharePoint with 
the new Business Connectivity 
Services. 


SOLUTIONS! Join Asif Rehmani 
and special guest speakers from 
our IT Pro and Developer tracks 
as they unveil the big-win 
solutions that SharePoint 
delivers, out-of-the-box. Learn 
to create high-value, no-code 
solutions with tools like 
SharePoint Designer, InfoPath, 
SharePoint Workspaces, Excel 
and Access Services, and Office 
Web Apps. Discover what you 
can do to automate processes 
and deliver the composite and 
collaboration solutions that your 
users are demanding. 


DEV 


HDEVO01: CREATING RICH BUSINESS 
APPLICATION WITH THE MANAGED CLIENT 
OBJECT MODELS IN SHAREPOINT 2010 
ANDREW CONNELL 


HDEV02: LOCAL DATA ACCESS IN SHAREPOINT 
2010: LINQ AND BEST PRACTICES 
ANDREW CONNELL 


HDEV03: INCORPORATING MANAGED 
METADATA IN CUSTOM SOLUTIONS 
ANDREW CONNELL 


HDEV04: DEVELOPING A CUSTOM CLAIMS 
PROVIDER TED PATTISON 


HDEV05: EXTENDING THE VISUAL STUDIO 
2010 SHAREPOINT TOOLS 
TED PATTISON 


HDEV06: REMOTE DATA ACCESS IN 
SHAREPOINT 2010 TED PATTISON 


HDEV07: ADVANCED EXTERNAL LISTS IN 
SHAREPOINT 2010 SCOT HILLIER 


HDEV08: CREATING SEARCH-BASED 
SOLUTIONS WITH SHAREPOINT 2010 
SCOT HILLIER 


HDEV09: BEST PRACTICES FOR 
SANDBOXED SOLUTIONS 
SCOT HILLIER 


HDEV10: CREATING CUSTOM LINE OF 
BUSINESS SOLUTIONS WITH BUSINESS 
CONNECTIVITY SERVICES 

TODD BAGINSKI 


HDEV11: BUILDING CUSTOM APPLICATIONS 
(MASHUPS) ON THE SHAREPOINT PLATFORM 
TODD BAGINSKI 


HDEV12: CREATING CUSTOM WORKFLOWS 
AND REUSABLE WORKFLOW ACTIVITIES FOR 
SHAREPOINT DESIGNER 

TODD BAGINSKI 


HDEV13: BUILDING CUSTOM APPLICATIONS 
WITH THE POWERPIVOT API 
MAURICE PRATHER 


HDEV14: BEST PRACTICES FOR UPGRADING 
WEB PARTS MAURICE PRATHER 


HDEV15: HOW TO BUILD CLAIMS-AWARE 
APPLICATIONS AND CONTROLS 
MAURICE PRATHER 


HDEV16: SHAREPOINT 2010 POWERSHELL 
FOR DEVELOPERS GARY LAPOINTE 


HDEV17: LEVERAGING THE SHAREPOINT 
2010 USER EXPERIENCE ENHANCEMENTS 
GARY LAPOINTE 
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HDEV18: EXTENDING THE SOCIAL 
EXPERIENCE USING THE SHAREPOINT 2010 
SOCIAL NETWORKING API 

GARY LAPOINTE 


IT PRO 


HITP01: BEST PRACTICES FOR LEAST- 
PRIVILEGE INSTALLATION, ADMINISTRATION, 
AND SECURITY OF SHAREPOINT 2010 

DAN HOLME 


HITP02: DESIGNING GOVERNANCE: HOW 
INFORMATION MANAGEMENT AND SECURITY 
MUST DRIVE YOUR DESIGN 

DAN HOLME 


HITP03: FILE SHARING SMACKDOWN: 
SHARES VS. SHAREPOINT 
DAN HOLME 


HITP04: INFORMATION ARCHITECTURE AND 
THE MANAGED METADATA SERVICE 
DAN HOLME 


HITP05: ARCHITECTING AND MANAGING 
VIRTUALIZED SHAREPOINT 2010 FARMS 
MICHAEL NOEL 


HITP06: SHAREPOINT 2010 DISASTER 
RECOVERY AND HIGH AVAILABILITY 
MICHAEL NOEL 


HITP07: MANAGING MULTIPLE 
AUTHENTICATION PROVIDERS IN 
SHAREPOINT 2010 FOR EXTRANETS 
MICHAEL NOEL 


HITP08: SHAREPOINT 2010 UPGRADE 
DRILL-DOWN 
JOEL OLESON 


HITP09: ADMINISTRATION OF SHAREPOINT 
2010 USING WINDOWS POWERSHELL, THE 
NEW COOLNESS 

SHANE YOUNG & TODD KLINDT 


HITP10: GETTING COZY WITH SERVICE 
APPLICATIONS 
SHANE YOUNG & TODD KLINDT 


HITP11: KEEPING AN EYE ON 
SHAREPOINT 2010 
SHANE YOUNG & TODD KLINDT 


HITP12: AUTHENTICATION CHANGES IN 
SHAREPOINT 2010 
SHANE YOUNG & TODD KLINDT 


HITP13: SHAREPOINT 2010 DEPLOYMENT 
DEMOFEST BEN CURRY 


HITP14: FARM ARCHITECTURE PLANNING 
AND PERFORMANCE TESTING 
BEN CURRY 


HITP15: WHAT DO YOU NEED FOR EFFECTIVE 
COMMUNICATION BETWEEN IT PROS AND 
DEVELOPERS? A REFEREE! 

BEN CURRY 


HITP16: SHAREPOINT 2010 SEARCH 
MATT MCDERMOTT 


HITP17: PLANNING AND DEPLOYING SOCIAL 
COMPUTING FOR SHAREPOINT 2010 
MATT MCDERMOTT 


HITP18: ENTERPRISE SOCIAL COMPUTING 
WITH SHAREPOINT 2010 
MATT MCDERMOTT 


NO CODE SOLUTION 


HNCS01: CREATING CONTENT-CENTRIC SITES 
WITH SHAREPOINT 2010 WEB CONTENT 
MANAGEMENT ANDREW CONNELL 


HNCS02: CREATING BI SOLUTIONS WITH 
SHAREPOINT 2010 USING 
PERFORMANCEPOINT SERVICES 

TED PATTISON 


HNCS03: USING OUTLOOK AND THE 
SHAREPOINT WORKSPACE WITH 
SHAREPOINT 2010 SCOT HILLIER 


HNCS04: VISUALLY CREATING VISUALLY 
COMPELLING WORKFLOWS (WITHOUT 
WRITING ANY CODE!) TODD BAGINSKI 


HNCS05: LEVERAGE EXCEL SERVICES TO 
DRIVE OTHER WEB PARTS WITHOUT CODE! 
MAURICE PRATHER 


HNCS06: USE DATA VIEWS TO GET TO YOUR 
DATA — BOTH INSIDE AND OUTSIDE OF 
SHAREPOINT ASIF REHMANI 


HNCS07: AUTOMATING BUSINESS PROCESSES 
USING INFOPATH 2010 FORMS WITH 
INTEGRATED SHAREPOINT DESIGNER 2010 
WORKFLOWS ASIF REHMANI 


HNCS08: MANAGE YOUR EXTERNAL DATA 
USING BUSINESS CONNECTIVITY SERVICES ... 
WITHOUT CODE! ASIF REHMANI 


HNCS09: USING INFOPATH 2010 AND 
SHAREPOINT DESIGNER 2010 TO MANAGE 
SHAREPOINT LIST FORMS ASIF REHMANI 


Also in-depth sessions delivered 
by Microsoft's SharePoint team to 
be posted on our website July 1st. 


DOoTNETNUKE 
CONNECTIONS 


DEPLOYING DOTNETNUKE WEB SITES AS A 


KEYNOTE 

COMPLETE SOLUTION 
DOTNETNUKE CONNECTIONS IAN ROBINSON 
'10 KEYNOTE ADDRESS 


FROM DEV TO STAGING TO PRODUCTION: 
ENTERPRISE DEPLOYMENT SCENARIOS 
IAN ROBINSON 


DOTNETNUKE PERFORMANCE: 


SHAUN WALKER 


SKIN DEVELOPMENT 


DOTNETNUKE FOR MOBILE DIAGNOSING PROBLEMS 

BEATRIZ OLIVEIRA MITCHEL SELLERS 

SUPER STYLESHEETS CONFIGURING DOTNETNUKE FOR 

BEATRIZ OLIVEIRA PERFORMANCE IN A WEB FARM 

CREATING A RICH USER EXPERIENCE WITH NIGIS CHRISTY 

TELERIK COMPONENTS DOTNETNUKE 5 ADMINISTRATION: TIPS 
CUONG DANG AND TRICKS 

THE FUTURE OF DOTNETNUKE DESIGN WITH ME 

HTML5 AND CSS3 ROI: COMMUNITY IS GOOD BUSINESS 
CUONG DANG SCOTT WILLHITE AND JOE BRINKMAN 


DOTNETNUKE CORP. — UNDERLYING 
PHILOSOPHY & BUSINESS VISION 
NAVIN NAGIAH 


CLOSING PANEL: THE ROAD AHEAD 
SHAUN WALKER, SCOTT WILLHITE, 
JOE BRINKMAN, ROB CHARTIER 


DESIGN (SKIN DEVELOPMENT) 


CSS GRID FRAMEWORK SYSTEMS 
JENNI MERRIFIELD 


DEVELOPMENT 


EFFECTIVE AUDITING AND LOGGING IN 
DOTNETNUKE MODULES 
BRANDON HAYNES 


SECURE MODULE DEVELOPMENT 
BRANDON HAYNES 


BUILDING FRIENDLY URLS INTO 
DOTNETNUKE MODULES 
BRUCE CHAPMAN 


HOW TO LOCALIZE YOUR DOTNETNUKE SITE 
CATHAL CONNOLLY 


PROFILING YOUR DOTNETNUKE MODULE FOR 
MAXIMUM PERFORMANCE 
KEIVAN BEIGI 


DEMYSTIFYING DOTNETNUKE 5 EXTENSION 
DEVELOPMENT 
MITCHEL SELLERS 


MODULE DEVELOPMENT 


MVP STYLE MODULE DEVELOPMENT - 
DEEP DIVE 
CHARLES NURSE 


COMPLETE PERFORMANCE TUNING IN THE 
DOTNETNUKE ENTERPRISE 
KRISTIAN RANSTROM 


ADMINISTRATION 


CREATING A NEW DOTNETNUKE INSTALL IN 
THE CLOUD 
BRUCE CHAPMAN 


LEVERAGING SNOWCOVERED.COM: BOOST 
SALES AND DECREASE SUPPORT ISSUES 


CAO NASH More sessions are under 


wraps. Check our Web site 
for more updates. 
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Penton Media 
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and save $100 off conference registration! 


LISTING 2: UPDATE Statement for 
ServerList SSIS 
UPDATE serverlist SSIS 
SET version = SQL Servers.Product Version Int 


FROM ServerList SSIS INNER JOIN SQL Servers ON 
ServerList SSIS.Server = SQL Servers.Server 


LISTING 3: Sample Query Using the 
Updated Version Field 

SELECT LTRIM(RTRIM(Server)) AS servername 

FROM ServerList SSIS 


WHERE (Connect = 1) AND (Version >=9) 
AND (DMZ = Ø) AND ONLINE = 1 


The Product_Version_int field is populated 
when the package executes the Load Server Info 
object. The value for this field is derived from the 
code 


@@microsoftversion / POWER(2, 24) 


When this code is executed against a server with a 
SELECT statement, it returns the value of 8 for SQL 
Server 2000, 9 for SQL Server 2005, and 10 for SQL 
Server 2008. With the version information captured in 
this manner, there’s no need to parse strings of text. 

Updating the SSIS package. To update the SSIS 
package to use the Product_Version_Int field, I 
added the Update ServerList_SSIS Product Version 
task. Figure 3 shows this task, which is annotated to 
explain the flow. This figure also shows the query that 
updates the Version field in the ServerList_SSIS table 
with the value in Product_Version_int. Listing 2 also 
shows this query in case you'd like to inspect it more 
closely. 

A sample query that uses the updated Version 
field is shown in Listing 3. In this example, the query 
is selecting servers running SQL Server 2005 or later. 


Parallel Processing 

In the original solution, each task executed sequen- 
tially. (The one exception was the initial truncation 
of tables.) This design worked well enough for 30 
or fewer servers but eventually showed kinks in the 
armor when more servers had to be polled, espe- 
cially when the polling took place over slower net- 
work links. To improve performance, changes need 
to be made. 

In addition to consolidating some data flows, I 
decided to incorporate parallel processing in parts 
of the SSIS package. Fortunately, like the other 
modifications, it was fairly easy. Instead of using 
individual ForEach Loop containers for the data 
flows that gathered and consolidated information 
from each server, I grouped all the data flows into 
one ForEach Loop container. 
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Prior to this modification, it took an average 
of 3 minutes for the SSIS package to poll 31 local 
and remote SQL Server instances. (These servers 
represent a small subset of a much larger infra- 
structure.) I was hoping that parallel processing 
would reduce the execution time by at least 20 
percent. 


The Test 

With all three modifications complete, I was ready 
to test the DBA Repository 2010 against the origi- 
nal one for comparison purposes. I first tested the 
original solution. While watching the execution in 
design mode, I saw the inefficiencies exacerbate. For 
31 servers, it took 2 minutes and 44 seconds to com- 
plete on the first run. 

With fingers crossed, I then executed the 
updated package. As might be expected with any 
design modification, there were small glitches 
that had to be overcome. l'll spare you the 
details, but suffice it to say that two new vari- 
ables and connections to support parallel pro- 
cessing were required. The variables and connec- 
tions were added because at some stages in the 
execution, the SSIS package is polling the SQL 
Server 2005 and SQL Server 2000 servers at the 
same time, while at other stages, the package is 
polling them separately. If the variables’ values 
change in the package, which they do many hun- 
dreds of times, they need to be isolated to their 
own ForEach Loop container. Otherwise, que- 
ries can suffer from syntax errors due to differ- 
ences between the schemas of the various SQL 
Server versions. 

After making the necessary tweaks, I put DBA 
Repository 2010 to the test again. The results did 
indeed meet my expectations. The time it took to 
poll the same 31 servers for the same information 
was I minute and 41 seconds. I was hoping for a 
20 percent improvement in processing time but 
realized an improvement of more than 35 percent 
on the first execution. Over several executions 
I noticed time variances, but for the most part, the 
times consistently showed more than a 20 percent 
improvement. 

It is important to note that while the per- 
formance increase is worthwhile, the updated 
SSIS package is only incorporating limited par- 
allel tasks. It's possible to further parallelize 
individual tasks, and I welcome you to do so 
using the techniques covered herein. However, 
there's a point where contention for local object 
resources (table locking, for example) becomes 
an Issue. Adding more parallel tasks would also 
require additional variables and ForEach Loop 
containers to accommodate the simultaneous 
processing. 
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How to Use DBA Repository 

2010 

If you'd like to try DBA Repository 2010, you 

can download it by going to www.sqlmag.com, 

entering 125240 in the InstantDoc ID text box, 
clicking Go, then clicking the Download the Code 

Here button. If this is your first installation of 

the DBA Repository, you also need to download 

the code from “Use SSRS and SSIS to Create a 

DBA Repository" (February 2008, InstantDoc ID 

97840) and follow the installation instructions in 

the Code Install Instructions.txt file to create the 

DBA Rep database. 

After the DBA Rep database is created, exe- 
cute the DBA Repository Update 2010.sql file 
included in the download for this article. The 
DBA Rep database will now contain all the tables 
you need to use DBA Repository 2010, including 
the ServerList SSIS table and the Server Location 
table. There are only a few other steps you need to 
take to ensure a successful execution of the SSIS 
package: 

1. Manually populate the Server Location table. 
This table identifies the cities where your servers 
reside. It consists of only two columns: Location and 
LocationID. In the Location column, enter the name 
of each city. In the LocationID column, enter the 
number you want to represent that city. 

2. Manually populate parts of the ServerList_ 
SSIS table with your specific server information. 
For each server, you need to fill in the following 
fields: 

e Server. Enter the server’s name. 

e Connect. If you want to include the server in the 
polling list, enter a value of 1. If you don’t want 
to include the server in the polling list, enter a 
value of 0. 

e DMZ. If the server is in a DMZ, enter a value 
of 1. If the server is in a non-DMZ, enter a 
value of 0. 

e LocationID. Enter the number that represents the 
city where this server is located. (This should be 
the number you associated with that city in the 
Server_Location table.) 


The remaining two fields—Version and Online— 
are updated automatically when the SSIS package 
executes. 

3. Open the SSIS package using Business Intel- 
ligence Development Studio (BIDS) and enter the 
account information for your DMZ Connection 
Manager objects. The package assumes that you 
have a SQL Server authenticated admin account 
with the same password. If that isn’t the case, 
then only servers that can use the same account 
information should be added as DMZ servers 
in the ServerList_SSIS table. The package has 


a ProtectionLevel of DontSaveSensitive, so you 
need to modify this to either EncryptSensitive- 
WithUserKey or EncryptSensitiveWithPassword 
in the package’s properties to be able to edit the 
DMZ Connection Manager objects. Enter a valid 
server, user, and password and save this informa- 
tion in the package. You only need to perform this 
step one time for each DMZ connection to save 
the password and execute the package successfully 
henceforth. 

4. Enter the email address of the DBA to 
whom you want to send the offline-server notifica- 
tion. You enter this email address in the Send Mail 
of Offline Servers Execute SQL tasks. There are 
two of these in the package—one for DMZ and 
one for non-DMZ. (In a future version, I might 
use a variable to hold the email address so that it 
has to be entered only once.) The code that sends 
the offline-server notification (Listing 1) uses the 
sp_send_dbmail stored procedure to send the mes- 
sage about any offline servers. Therefore, you must 
have database mail configured on the SQL Server 
machine you're using to store the local DBA_Rep 
database. 


After these preparations, you can run the SSIS 
package. When it finishes, you'll find numerous 
tables containing the information that it gath- 
ered. You can then use SQL Server Reporting 
Services (SSRS) to query that repository data 
and create customized reports. The "Putting the 
Data to Use with Reporting Services" section 
in the article *Use SSRS and SSIS to Create a 
DBA Repository" contains an example of how 
to create a report about SQL Server drive space. 
This report indicates which drives are potentially 
getting low on free space on each server. It also 
specifies the database file sizes on each drive on 
each server. 


The Next Update 

The modifications I made to my original solution 
have added much needed functionality, and it's my 
hope you'll find DBA Repository 2010 useful. At 
the very least, I hope that some of the techniques I 
discussed will prove helpful to you. 

With these updates out of the way, Im going 
to start looking at ways to improve the analysis 
and reporting of the collected information. Who 
knows, I might even build a data mining solution 
with Key Performance Indicator (KPIs) and trend- 
ing for data growth over time. Just as when I first sat 
down three years ago to put this solution together, 
the ideas for improvement seem limitless. Feel free 
to email me with any ideas you have for modifying 
or expanding it. SOL 
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Parallelism Enhancements 
SQL Server 2008 


Upgrade to boost query performance 


QL Server 2008 introduces several interesting 

enhancements concerning parallel query pro- 

cessing. These enhancements mainly improve 
the performance of large-scale queries, such as 
queries against large data warehouses and queries 
against partitioned tables. In this article I describe 
three enhancements: few-outer-row optimization, 
star join optimization, and partitioning enhance- 
ments. All my examples use a sample database called 
testparallel. Run the code in Listing 1 to create this 
database, as well as a helper table function called 
GetNums that accepts a number as input and returns 
a sequence of integers in the range 1 through the 
input number. 


Few-Outer-Row Optimization 
The few-outer-row scenario refers to a nested loop 
join whose outer input (i.e., the top one) is a parallel 
scan that filters data based on a predicate and yields a 
small number of rows. Regardless of the join, a parallel 
scan uses a parallel page supplier that provides a set of 
pages to each thread upon request. As soon as a thread 
finishes processing a set of pages, it requests another set 
of pages. This way threads that process rows faster will 
process more rows and all threads should finish more 
or less at the same time. But after applying the filter as 
part of the scan, there's no assurance that the number 
of qualifying rows in each thread will be similar. In 
fact, you can end up with some threads that have zero 
qualifying rows and some with nonzero numbers. 

The inner part of a nested loop join isn't aware 
that it is processed in parallel; each thread simply 
processes whichever rows that it obtains from the 
upper part. If the upper part has multiple threads, 
this determines how many threads will be used to 
process the inner part, as well as how many outer 


Let's consider an example that demonstrates this 
imbalance in SQL Server 2005 and then the resolution 
to the imbalance in SQL Server 2008. I used the code 
in Listing 2 to create the sample data for the example. 
This code creates two tables called T1 and T2 with a 
million rows each, both with a clustered index on coll, 
and no index on col2. 

The following query has a join and a filter; it is 
processed using a nested loop join and fits the few- 
outer-row scenario: 
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SELECT * 
FROM dbo.T1 

JOIN T2 ON T1.coll = T2.coll 
WHERE Tl.col2 <= 100; 


FSQL Fundamentals (Microsoft Press). 


Figure 1 shows the plan I got for this query 
on SQL Server 2005 SP3. Note that the 
machine I used to test this code has eight 
logical CPU cores. 

Observe in the plan (by looking at the 
properties of the scan) that besides the main 
thread (thread 0), eight threads are used to process the 
parallel scan, but only one has qualifying rows (100 of 
them), whereas all the rest have zero qualifying rows. 


LISTING I: Code to Create Testparallel Sample 
Database and GetNums Helper Function 


-- Sample datatabase 

SET NOCOUNT ON; 

IF DB ID('testparallel') IS NULL CREATE DATABASE testparallel; 
GO 

USE testparallel; 

GO 


-- Helper function GetNums 
-- returns a sequence of integers of a requested size 
IF OBJECT ID('dbo.GetNums', 'IF') IS NOT NULL 

DROP FUNCTION dbo.GetNums; 


. . . . GO 
rows each thread will process in the inner part. Prior CREATE FUNCTION dbo.GetNums(@n AS BIGINT) RETURNS TABLE 
to SQL Server 2008, if the outer input to a nested EN 
loop join was the result of a parallel scan with uneven WITH 


distribution of qualifying rows among threads, there 
was no attempt made by the optimizer to somehow 
redistribute the rows to balance the work. This could 
result in imbalanced distribution of the load among 
threads, with some threads remaining idle and others 
doing most of the work. 
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LØ AS(SELECT 1 AS c UNION ALL SELECT 1), 
L1  AS(SELECT 1 AS c FROM LØ AS A CROSS JOIN LØ AS 
L2  AS(SELECT 1 AS c FROM L1 AS A CROSS JOIN L1 
L3 —AS(SELECT 1 AS c FROM L2 AS A CROSS JOIN L2 
L4  ASCSELECT 1 AS c FROM L3 AS A CROSS JOIN L3 
L5  ASCSELECT 1 AS c FROM L4 AS A CROSS JOIN L4 AS B), 
Nums AS(SELECT ROW NUMBER() OVER(ORDER BY (SELECT NULL)) AS n FROM L5) 
SELECT TOP (Qn) n FROM Nums ORDER BY n; 

GO 


AS 
AS 
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This distribution of rows unfortunately dictates the 
distribution of work among threads to process the 
inner part of the join, so only one thread ends up 
working and the other seven remain idle. 

SOL Server 2008 introduces an enhancement 
called few-outer-row optimization, in which the 
optimizer detects the few-outer-row scenario, and 
when detected, adds a Redistribute Streams operator 
above the scan to redistribute the rows evenly among 
threads. Then the work is distributed evenly among 
threads to process the inner part of the nested loop 
join. Figure 2 shows the plan I got for this query on 
SQL Server 2008 SP1. Note the uneven distribution 


LISTING 2: Code to Create Sample Data 
for Few-Outer-Row Optimization Example 


USE testparallel; 
IF OBJECT ID('dbo.T1') IS NOT NULL DROP TABLE dbo.T1; 
IF OBJECT ID('dbo.T2') IS NOT NULL DROP TABLE dbo.T2; 


GO 


CREATE TABLE dbo.T1 
C 


coll INT NOT NULL, 
col2 INT NOT NULL, 
filler BINARY(1ØØ) NOT NULL DEFAULT(Øx) 


DE 
CREATE UNIQUE CLUSTERED INDEX idx cl coli ON dbo.Ti(col1); 


INSERT INTO dbo.T1 WITH(TABLOCK) (coll, col2) 
SELECT n AS coll, n AS col2 
FROM dbo.GetNums (1000000) ; 


SELECT * INTO dbo.T2 FROM dbo.T1; 
CREATE UNIQUE CLUSTERED INDEX idx cl coli ON dbo.T2(col1); 


GO 
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Cost: 98 % 


of qualifying rows returned from the scan, but the 
redistribution of the rows evenly among threads by 
the Redistribute Streams operator. 


Star Join Optimization 

Star join optimization is mainly applicable to join 
queries against data warehouses. The classic data 
model in a data warehouse is known as the star 
schema model. In this model you have multiple 
dimension tables that hold information about 
the subjects that you analyze the data by (e.g., 
customer, employee, product, time) and a centric 
table called the fact table, with the facts—several 
measures for each applicable combination of 
dimension keys. Each dimension table typically has 
a compact surrogate key used in the fact table as a 
foreign key referencing the corresponding dimen- 
sion table. The dimension tables are typically fairly 
small (comparably), whereas the fact table can get 
quite large—many millions, or in some cases bil- 
lions, of rows. 

Classic queries against data warehouses with a 
star schema model involve a join between the big 
fact table and some of the dimension tables, with 
the join conditions based on the single column 
foreign key-primary key relationships, and some 
filters on nonkey attributes from the dimension 
tables. 

SQL Server processes a join between two inputs 
at a time. In the star join scenario, each join could 
end up processing a very large number of rows and 
therefore be inefficient. 

Let's consider an example of such a join to see 
how it is processed in SQL Server 2005, as well 
as SQL Server 2008's improvements. The code in 
Listing 3 creates the sample data for my examples. 
This code creates a general form of a data warehouse 
with a star schema model. It creates three dimension 
tables called Dim1, Dim2, and Dim3 and a fact table 
called Fact. The code fills the dimension tables with 
100, 50, and 200 rows, respectively, and the fact table 
with 1,000,000 rows. Following is an example of a 
classic star join query: 


SELECT * 
FROM dbo.Fact AS F 
JOIN dbo.Dim2 AS D2 
ON F.key2 = D2.key2 
JOIN dbo.Dim3 AS D3 
ON F.key3 = D3.key3 
WHERE D2.attrl «- 3 
AND D3.attrl <= 2; 


As you can see, the query joins the fact 
table with two of the dimension tables 
based on the foreign key-primary key 
relationships, and it filters the dimension 
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LISTING 3: Code to Create Sample Data for Star Join Optimization Example 


USE testparallel; 


key2 INT NOT NULL CONSTRAINT FK_Fact_Dim2 FOREIGN KEY 


IF OBJECT ID('dbo.Fact', 'U') IS NOT NULL DROP TABLE dbo.Fact; REFERENCES dbo.Dim2, 

IF OBJECT ID('dbo.Diml', 'U') IS NOT NULL DROP TABLE dbo.Dim1; key3 INT NOT NULL CONSTRAINT FK Fact Dim3 FOREIGN KEY REFERENCES dbo.Dim3, 
IF OBJECT ID('dbo.Dim2', 'U') IS NOT NULL DROP TABLE dbo.Dim2; measurel INT NOT NULL, 

IF OBJECT ID('dbo.Dim3', 'U') IS NOT NULL DROP TABLE dbo.Dim3; measure2 INT NOT NULL, 


GO 


CREATE TABLE dbo.Dim1 
C 
keyl INT NOT NULL CONSTRAINT PK Dimi PRIMARY KEY, 
attr1 INT NOT NULL, 
filler BINARY(1ØØ) NOT NULL DEFAULT (Øx) 
5 


CREATE TABLE dbo.Dim2 
K 
key2 INT NOT NULL CONSTRAINT PK_Dim2 PRIMARY KEY, 
attr1 INT NOT NULL, 
filler BINARY(1ØØ) NOT NULL DEFAULT (£x) 
DE 


CREATE TABLE dbo.Dim3 
& 
key3 INT NOT NULL CONSTRAINT PK Dim3 PRIMARY KEY, 
attri INT NOT NULL, 
filler BINARY(1ØØ) NOT NULL DEFAULT (0x) 


measure3 INT NOT NULL, 

filler BINARY(1ØØ) NOT NULL DEFAULT (x), 

CONSTRAINT PK Fact PRIMARY KEY(keyl, key2, key3) 
); 


NSERT INTO dbo.Diml(keyl, attr1) 
SELECT n, ABS(CHECKSUM(NEWIDO)) % 20 + 1 
FROM dbo.GetNums (100) ; 


NSERT INTO dbo.Dim2(key2, attrl) 
SELECT n, ABS(CHECKSUM(NEWID())) % 18 + 1 
FROM dbo.GetNums(5Ø); 


NSERT INTO dbo.Dim3(key3, attrl) 
SELECT n, ABS(CHECKSUM(NEWID())) % 4Ø + 1 
FROM dbo.GetNums (200) ; 


NSERT INTO dbo.Fact WITH (TABLOCK) 

(keyl, key2, key3, measurel, measure2, measure3) 
SELECT N1.n, N2.n, N3.n, 

ABS(CHECKSUM(NEWID())) % 1000000 + 1, 


5 ABS(CHECKSUM(NEWID())) % 1000000 + 1, 
ABS(CHECKSUM(NEWID())) % 1000000 + 1 
CREATE TABLE dbo. Fact FROM dbo.GetNums(1ØØ) AS N1 
( CROSS JOIN dbo.GetNums(50) AS N2 
keyl INT NOT NULL CONSTRAINT FK Fact Diml FOREIGN KEY CROSS JOIN dbo.GetNums(200) AS N3; 
REFERENCES dbo.Diml, GO 
rows by nonkey attri- 3 m 2d E 
: 7 SELECT Parallelism Hash Match Parallelism Clustered Index Scan 
butes. Figure 3 shows : (Gather Streams) (Inner Join) (Distribute Streams) [0im2].[PK.0im2] [D2] 
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the execution plan I got 
al >» 
for this query on SQL Hash Match Parallelism Clustered Index Scan 
Server 2005 — (Inner Join) (Distribute Streams) [Dim3].[PK Dim3] [03] 
x - id Cost: 9 X Cost: O X Cost: O X 
All 1,000,000 rows T" 


are returned from the 
scan of the clustered 


Clustered Index Scan 
[Fact].[PK Fact] [F] 


1000000 rows Cost: 88 X 


index on Fact and are Figure 3 
used to probe the hash 
table created based on 
the rows retrieved from 
the clustered index on Dim3. This means that the join 
has a lot of rows to process. The result of this join 
is still fairly large—85,000 rows. This result is used 
to probe the hash table created based on the rows 
retrieved from the clustered index on Dim2. 

SQL Server 2008 introduces an enhancement 
called star join optimization. It uses heuristics to 
detect a star join (e.g., minimum size of fact, single- 
column joins); when detected, it can add so-called 
optimized bitmap filters to the plan. Think of a 
bitmap filter as a compact in-memory representa- 
tion of a fairly small set of values that can be used 
in the plan to filter data. When the optimizer detects 
a star join scenario, it evaluates the use of bitmaps 
to produce a compact in-memory representation of 
the qualifying dimension keys after the filtered scan 
of each dimension. Then, later in the plan when 
scanning the fact table, it can apply a number of 
filters based on those bitmaps filtering out the bulk 
of the rows from the fact table. Then what's left for 
the join operators to process are much-reduced sets 
of rows. 
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Star join (SOL Server 2005) 


SQL Server 2005 does support bitmap filters in 
certain cases. What's new in SQL Server 2008 is sup- 
port for the so-called optimized bitmap filters. The 
optimized filters can be added dynamically during 
optimization when a star join scenario is detected, 
and the optimizer can rely on estimates based on the 
optimized filters to be able to make educated decisions 
in later parts of the plan tree. Optimized bitmap filters 
can be used only in parallel plans with hash joins. 
Figure 4 shows the plan I got for our star join query 
on SQL Server 2008. 

A small number of rows are returned after the fil- 
tering of each dimension. In both cases, the Distribute 
Streams operator creates bitmaps (called Opt | 
Bitmap1007 and Opt. Bitmap1006) based on the quali- 
fying keys and broadcasts those bitmaps to all threads. 
The qualifying rows in both cases are used as the build 
inputs to the hash tables. The fact table is scanned. 
From the Predicate used to filter the rows in the scan 
you can observe that both bitmaps were used to filter 
the fact table rows. After applying both filters, a small 
number of rows remain— $8,001 in this case. This means 
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Star join (SOL Server 2008) 


LISTING 4: Code to Create Sample Data for 
Partitioning Example 
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Predicate 

PROBE([Opt Bitmap1006) {testparatiel) (dbo) [Fact] (key3] 

as [FL Ikey3).NTIN ROW]) AND PROBE(TOpt. Bitmap1007], 

Testparallei] [Gbo] Facti {key2] as (FI-IkeyZ] NIN ROW) 

Object 

[testparatiel} {dbo} [Fact] [PK Fact) [F] 


that the number of rows left for the joins to process is 
much smaller compared with the plan in SQL Server 
2005, in which star join optimization wasn't used. The 
way you can tell that the new optimized bitmap filters 
were used is by their name: They have the prefix Opt . 


Partitioning 

The last enhancement is an improvement in the par- 
allel processing of queries against partitioned tables. 
To demonstrate this enhancement, run the code in 
Listing 4 to create and populate a partitioned table 
called PartitionedTable, then query the table. The 
code in Listing 4 populates the table with 1,000,000 
rows in four partitions (with coll ranges <= 250,000; 
> 250,000 and <= 500,000; > 500,000 and <= 750,000; 
> 750,000). 

In SQL Server 2005, as long as your query has 
only one qualifying partition, parallel query pro- 
cessing usually performs good distribution of the load 
among threads. As an example, the following query 

filters rows from only one partition: 


I == Nw 1 m 
SELECT 3 | Filter Sort 
lost: 0 X (Gather Streams) Cost: O X Cost: 53 X 


Cost: 19 X 


Clustered Index Scan 
[PartitionedTable].[idx c 
Cos 28 * 


SELECT * 

FROM dbo.PartitionedTable 
WHERE coll «- 250000 
ORDER BY col2; 


Figure 5 shows the plan I got for this query on 
SQL Server 2005. Observe in the plan that there's 
a good distribution of the load among the eight 
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Figure 6 
Multiple partitions (SOL Server 2005) 
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is smaller than the query degree of parallelism 
(DOP), some of the threads will remain idle. As 
an example, in the following query two partitions 
qualify: 


SELECT * 

FROM dbo.PartitionedTable 
WHERE coll <= 500000 
ORDER BY col2; 
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Figure 6 shows the execution plan I got for this 
query on SQL Server 2005. The Constant Scan SELECT 
operator enumerates the qualifying partitions, and 
then one thread is assigned to work on each parti- 
tion via the Nested Loops operator. You can see 
from the properties of the Clustered Index Seek in 
the inner part of the Nested Loops join that only 
two threads actually worked. As you can imagine, 
the distribution of work among threads in such 


cases can be inefficient. 
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Figure 7 
Multiple partitions (SOL Server 2008) 


SQL Server 2008 doesn't usea thread-per-partition 
strategy in case multiple partitions qualify. It also 


doesn't use the Constant Scan 
and Nested Loops operators 
to enumerate the partitions 
and issue the work per parti- 
tion. In plans involving par- 
titioned tables, you'll see the 
work as if done against one 
partition; from the operator 
properties you can determine 
how many and which parti- 
tions were processed. In terms 
of parallelism, regardless of 
the number of qualifying par- 
titions, SQL Server distributes 
the work among all threads. 
As an example, Figure 7 
shows the plan I got on SQL 
Server 2008 for the last query. 
As you can see, in SQL Server 
2008 the work was distrib- 
uted nicely among all threads 
even though two partitions 
qualified. 


Built-In Boost 
Allthe enhancements I discussed 
—few-outer-row optimizations, 
star join optimizations, and 
improvements in parallel 
query processing against 
partitioned — tables—are 
built-in improvements in the 
engine, mainly improving 
the performance of large- 
scale queries. None of these 
enhancements require any 
sort of intervention from 
the administrator's side—no 
need for any knobs, switches, 
or query revisions. You get a 
performance boost for your 
existing queries simply by 
upgrading to SQL Server 
2008. En 
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A good SQL 
programmer must 
develop the mental 
discipline to explore 
set-based possibilities 
thoroughly before falling 
back on the intuitive 
procedural solution. 


s T-SQL programmers, we always hear that 
A the SQL language is optimized for set-based 

solutions rather than procedural solutions, 
but we seldom see examples from that perspective. 
Consequently, many beginning SQL programmers 
don't have a clear understanding of what set-based 
means in terms of the code they need to write to solve 
a specific problem. 

Even for those who understand the concept, there 
are many programming problems for which a set-based 
solution seems impossible. Sometimes that's true. It's 
not always possible to find a set-based solution, but 
most of the time we can find one by using a little cre- 
ative thinking. A good SQL programmer must develop 
the mental discipline to 
explore set-based possi- 
bilities thoroughly before 
falling back on the intui- 
tive procedural solution. 

In this article, I provide 
a relatively simple example 
thatillustrates how to think 
in a set-based way about a 
common type of problem 
that also has an intuitive 
procedural solution. 


The Business 
Case 
When you visit the doc- 
tor's office, the first thing 
the nurse does is put you on a scale, record your weight, 
and check your height. Checking your weight makes 
sense from a medical point of view, but have you ever 
wondered why the nurse records your height each time? 
Unless you're very young, your height hasn't changed 
since your last visit and isn't likely to change again. 
The reason the nurse checks your height is to guard 
against identity theft. Health care providers want to 


make sure that the services they provide are going to 
the person who gets the bill—not to an imposter with 
a forged identity card. 

This kind of identity theft happens more frequently 
than you might think. HIPAA regulations now require 
an audit of changes in permanent physical characteristics 
in a patient's history that might suggest identity theft. 

Querying this kind of information provides a good 
example for comparing procedural thinking and set- 
based thinking when programming in SQL. 


The Problem Statement 

The generic programming problem is that the solution 
depends on the order of rows and requires the com- 
parison of current row values with values in previous 
rows. This is a type of problem in which the procedural 
solution is intuitive, but the set-based solution isn't so 
obvious. 

In this particular problem, we're looking for rows 
where a previous visit for the same patient has a height 
value that's different from the height on the current 
record. We want to return the patient's unique medical 
record number, the date the change occurred, what the 
height was changed from, and what the height was 
changed to. We don't want to return any records that 
don't mark a change in height. 

Listing 1 gives you the code to create and popu- 
late the tables in this example, if you'd like to run the 
example yourself. 


A Procedural Approach 

The intuitive, procedural way to attack this problem is 
to order the records by patient and visit date, then loop 
through the records for each patient one row at a time. 
We query the first record for the patient and save the 
patient's original height in a variable. Then, we loop 
through subsequent records for the patient, comparing 
height values. If we find that the height is different on 
a subsequent record, we write an audit record, update 
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the height variable with the current value, and continue 
looping through the rows. Then we move to the next 
patient. 

Listing 2 contains the code for the cursor-based 
solution. The cursor method works, but its very 
inefficient. It could pose a serious performance 
problem when working with a large number of rows. 
How can we do this in a set-based, and presumably 
more efficient, way? 


A Set-Based Approach 

The difference between a procedural and set-based 
solution boils down to the way you define the 
problem. Stated in its simplest form, the change 
we're interested in involves only two records: two 
consecutive visits by the same patient. Everything 
else is irrelevant. 


The difference between 
a procedural and set- 
based solution boils 
down to the way you 
define the problem. 


We start by ordering the data by the patient's 
ID number and then by visit date. In that way, the 
records of consecutive visits by the same patient are 
adjacent to each other. The problem is then reduced 
to finding a way to join consecutive records from 
this set. 

When we understand the problem in that way, 
the solution isn't so difficult to discover. We need to 
create a sequence number for the sorted rows that 
can be used to join one record with the next in a 
self-join. 

We can create a common table expression 
(CTE) populated with patient data sorted by PatientID 
and VisitDate, adding a sequential ID using the 
ROW NUMBER(Q) function. We can self-join this 
temporary table like this: 


.. from CTE t1 
join CTE t2 on t2.ROWID = t1.ROWID + 1.. 


This will produce a set of records that represents every 
possible opportunity for the value of the patient’s 
height to change—that is, a set of records such that 
each contains the data from each set of two consecutive 
records in the original data set. 

At this point, filtering out the records that don’t 
represent a change is trivial. We simply review our 
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SET-BASED SQL 


LISTING I: Creating and Populating the Tables 
/* 


We use the AdventureWorks sample database to create tables for 
our test but you may use another database by changing 

the USE statements in all 3 listings. 

Sy 


USE AdventureWorks ; 
SET NOCOUNT ON; 


CREATE TABLE Dates 
(ID int, VisitDate datetime); 


--populate table with 20 visit dates 
DECLARE Qi int, Gstartdate datetime; 
SET Qi = 1; 

SET Gstartdate = GETDATEO ; 


WHILE Qi «- 20 
BEGIN 
INSERT Dates 
(ID, VisitDate) 
VALUES (Qi, Gstartdate); 


SET Gstartdate = DATEADD(dd,7, Gstartdate); 
SET Qi = @i+1; 
END 


CREATE TABLE PatientHeight 
(PatientID int not null 
,Height int); 


-- populate table with 1000 patientids with heights between 59 and 74 inches 
SET @i = 1; 


WHILE Gi <= 190000 

BEGIN 
INSERT PatientHeight 
(PatientID, Height) 
VALUES (Gi, Gi % 16 + 59); 


SET Qi = @i+1; 
END 


ALTER TABLE PatientHeight ADD CONSTRAINT PK PatientHeight 
PRIMARY KEY(PatientID); 


-- cartesian join produces 200,000 PatientVisit records 


SELECT 
ISNULL(PatientID, -1) AS PatientID, 
ISNULL(VisitDate, '19000101') AS VisitDate, 
Height 


INTO PatientVisit 
FROM PatientHeight 
CROSS JOIN Dates; 


ALTER TABLE PatientVisit ADD CONSTRAINT PK PatientVisit 
PRIMARY KEY(PatientID, VisitDate); 


-- create changes of height 
SET Q1 = 3; 


WHILE Gi < 10000 

BEGIN 

UPDATE pv 

SET Height = Height +2 

FROM PatientVisit pv 

WHERE PatientID - Qi 

AND pv.VisitDate = 

(SELECT TOP 1 VisitDate 

FROM Dates 

where id = ABS(CHECKSUM(Qi)) % 19); 


SET Qi = Qi + 7; 
END 


* 


-- return AdventureWorks to its previous state when you are finished 
-- with this example. 


DROP TABLE Dates; 

DROP TABLE PatientHeight; 
DROP TABLE PatientVisit; 
ny 


statement of the problem: To qualify as a record of 
interest, the patient must be the same in consecutive 
visits but the two heights must be different. Listing 3 
contains the code that implements this set-based 
method. 
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LISTING 2: The Cursor-Based 
Solution 
USE AdventureWorks; 


CREATE TABLE #Changes 
( PatientID int 


, VisitDate datetime 

, BeginHeight smallint 

, CurrentHeight smallint); 
DECLARE GPatientID int 

5 @CurrentID int 

å GBeginHeight smallint 
í @CurrentHeight smallint 
" GVisitDate datetime; 


SET GPatientID = 0; 


DECLARE Patient cur CURSOR FAST FORWARD FOR 
SELECT PatientID 

, VisitDate 

, Height 

FROM PatientVisit 

ORDER BY PatientID 

,VisitDate; 


OPEN Patient cur; 


FETCH NEXT FROM Patient cur INTO 
@CurrentID, @VisitDate, @CurrentHeight; 


WHILE @@FETCH_STATUS = 9 

BEGIN 

-- first record for this patient 

IF @PatientID <> GCurrentID 

BEGIN 
SET @PatientID = @CurrentID; 
SET @BeginHeight = 
@CurrentHeight; 

END 


IF @BeginHeight <> @CurrentHeight 
BEGIN 

INSERT #Changes ( PatientID 
, VisitDate 

, BeginHeight 

, CurrentHeight) 

VALUES 

(@PatientID 

, GVisitDate 

, GBeginHeight 

, @CurrentHeight) ; 


SET @BeginHeight = @CurrentHeight; 

END 

FETCH NEXT FROM Patient_cur INTO 
@CurrentID, @VisitDate, 
@CurrentHeight; 

END 


CLOSE Patient_cur; 
DEALLOCATE Patient_cur; 


SELECT * FROM #Changes 


DROP TABLE #Changes 


LISTING 3: The Set-Based 
Solution 


USE AdventureWorks; 
WITH PV_RN AS 


SELECT ROW_NUMBER() OVER (ORDER BY 
PatientID, VisitDate) AS ROWID, * 
FROM PatientVisit 


J 

select tl.PatientID 

,t2.VisitDate as DateChanged 

,tl.Height as HeightChangedFrom 

,t2.Height as HeightChangedTo 

from PV RN t1 

join PV RN t2 on t2.ROWID = t1.ROWID + 1 
where tl.patientid - t2.patientid 

and tl.Height «» t2.Height 
order by tl.PatientID, t2.VisitDate; 
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TABLE |: Execution Statistics 


[Method | Execution | Duration | Reads | CPU | 


Set-Based Cold 503 1298 515 
Cursor Cold 4090 203646 3931 
Set-Based Hot 416 1248 484 
Cursor Hot 3958 203728 | 3713 


Relative Performance of the 
Two Methods 

In Listing 1, we created the PatientVisit table and popu- 
lated it with 200,000 records containing the PatientID, 
VisitDate, and the Height recorded for that visit. The 
table contains about 2,600 records that represent a 
change in height for a patient. 

We used SQL Profiler to capture execution statis- 
tics of the two methods. First, we flushed the buffers 
to get the cold execution statistics, then we re-ran the 
query to get hot execution statistics after the data 
was in cache. Both the cursor and the set-based code 
returned identical results. Table 1 shows the execution 
statistics for each. Notice 
the huge difference in logical 
reads. This 160:1 difference 
can be a show stopper in 
many situations. CPU and 
Duration are roughly eight 
times as high in the cursor 
solution. 

The auditing require- 
ments for a large healthcare 
provider can easily generate 
a million rows per day in 
the audit table. So, even if 
you run your audit reports 
for only a single day's data, 
you'll have a lot of rows to 
process—far too many for a cursor or other looping 
mechanism to handle efficiently. 


thought. 


Set-Based Thinking 

Note that the more efficient solution operates on 
whole sets of data, not on the individual rows. Com- 
pare this with the cursor solution, in which operations 
are repeated for each row in a set. 

Nothing in this simple example is rocket science. 
You'll encounter SQL problems that are much more 
difficult to solve in a set-based way and some that are 
impossible. However, even this example requires a 
significant mental adjustment for programmers new 
to SQL programming. It requires a conscious effort to 
pull yourself out of your comfort zone and think in 
a new way. Even in the most difficult situations, don't 
give up on a set-based solution until you've given it a 
fair amount of thought. SOL] 

InstantDoc ID 125198 
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Even in the most 
difficult situations, 
don't give up on a 
set-based solution 
until you've given 
it a fair amount of 


Is Tempdb Affecting 


Your Day-to-Day 


Server 
rmance? 


Ba 


Some simple investigation is all it takes 


he tempdb database is essential to normal 
i SQL Server operation and is used extensively 
both by internal system-level SQL Server 
processes and by user-generated requests. In fact, 
tempdb is often the most active database in a SQL 
Server instance in terms of sheer numbers of trans- 
actions per second. I’m not just talking about those 
times when you create a temporary table using the # 
or ## prefix. Many other activities utilize tempdb, 
too—for example, sorting or hashing operations and 
intermediate results from joins. 

These types of operations differ from the activity 
in a regular user database in several key ways. First, 
objects such as temporary tables or internal work 
tables that support the aforementioned operations 
are typically created at a much higher rate than 
objects normally created in user databases. Second, 
there’s often an enormous number of pages that 
need to be allocated when populating these tempo- 
rary or work tables in a very short period of time. 
Together, these two activities can cause performance 
problems if not properly accounted for, and every 
DBA should be aware of them. 


Don’t Contend With Me 

One of the issues that arises from heavy tempdb 
use is contention amongst several of the pages in 
the data file that are used to track page allocations 
and usage. The two most common would be the 
Page Free Space (PFS) and Shared Global Alloca- 
tion Map (SGAM) pages, which reside in the first 
and third pages after the header page of every data 
file and are repeated about every 4GB as the file 
grows. There’s a lot of good information available 
on this subject, so I won’t dwell too much on the 
details, but in a nutshell, when you have many users 
attempting to create new tables or indexes at the 
same time, a bottleneck can occur on these pages 
because each new object creation requires an update 
to those pages. 


SQL Server Magazine * www.sqlmag.com 


By default, tempdb has only one data file—and 
hence a single set of these pages for a given size of 
the file. If this is a bottleneck in your system, you 
can usually spot it fairly easily by looking at the wait 
stats and, in particular, the PAGELATCH UP wait 
type. The higher the wait time, the more contention 
you have. 

You can confirm that the contention is in tempdb 
by looking at the Waitresource column in Master 
..Sysprocesses for resources of either 2:1:1 (PFS 
page) or 2:1:3 (SGAM page). The first number is 
the database ID (tempdb is always 2), the second 
is the data file ID, and the third is the page number 
in the file. See the “Managing Extent Allocations 
and Free Space" section of SQL Server 2008 Books 
Online (msdn.microsoft.com/en-us/library/ms175195 
.aspx) for more details. 

The most common way to reduce or alleviate this 
type of contention is to add more data files of equal 
size to the database, and thus spread the load across 
each of the files. Just don't add too many files at once! 
Add a few, then check the waits again to see whether 
you need more. Too many files can cause other prob- 
lems, so play it smart. The two white papers listed at the 
end of this article can help you determine how many 
files to use. 


Cache Me Out 

The second problem I want to address involves the 
way new temporary table creation can be affected 
by some caching techniques that help performance 
in tempdb in SQL Server 2005, along with the 
cleanup of the temporary tables. Itzik Ben-Gan's 
*Caching of Temporary Objects" (InstantDoc ID 
101993) goes into great detail about how the caching 
mechanism works and what can affect the caching. 
The bottom line is that SQL Server can cache some 
of the metadata and page allocations from tem- 
porary objects for easier and faster reuse with less 
contention. Again, since Itzik's article covers most 
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of the situations that prevent cache reuse, I won’t 
go into the details here, but like any caching mecha- 
nism it can dramatically speed up operations that 
take advantage of the cache. When used properly, 
it can make a significant performance difference on 
high-volume systems. 

But there’s another aspect not covered in Itzik’s 
article, and it relates to the explicit truncating and 
dropping of temporary tables in the users’ code. 
Although it doesn’t appear to be documented as 
such, I’ve confirmed that explicitly dropping a 
temporary table will essentially defeat the ability to 
use the caching mechanism. SQL Server implicitly 
cleans up temporary objects when they go out of 
scope, such as when the stored procedure is finished 
executing, so there’s little need to do it yourself 
anyway. Truncating the table when you're done with 
it can also be harmful to performance and should 
be avoided. 

To give you an example of the kind of impact 
this might have, I came across a situation in which 
someone was explicitly truncating and dropping the 
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temporary table at the end of the stored procedure. 
Iasked this person to try removing the Truncate and 
Drop commands, and the execution times decreased 
by over 50ms per execution. Because the user was 
executing these stored procedures hundreds of times 
per second, the removal resulted in a noticeable 
improvement in performance. There might be times 
when an explicit drop or truncate is necessary but they 
should be the exception rather than the rule. 


Spend a Few Minutes 
Checking for these common performance problems 
with tempdb is easy and can lead to better overall 
performance in your SQL Server instance. So, spend a 
few minutes optimizing tempdb by using this article's 
tips, along with other terrific information in two 
white papers about tempdb: "Optimizing tempdb 
Performance" (msdn.microsoft.com/en-us/library/ 
ms175527.aspx) and “Working with tempdb in SQL 
Server 2005" (technet.microsoft.com/en-us/library/ 
cc966545.aspx). E 
InstantDoc ID 125203 
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Services in 
SharePoint 2010 


Get a complete view of your business data with 
context-driven dashboards 


here used to be a product called Microsoft 
i Office Business Scorecard Manager. It had 
some compelling monitoring and analytic 
capabilities. As a successor to that product, Micro- 
soft released a product called Microsoft Office 
PerformancePoint Server in November 2007. 
PerformancePoint Server 2007 included moni- 
toring and analytic features, such as dashboards, 
scorecards, Key Performance Indicators (KPIs), 
reports, filters, and strategy maps, which were 
delivered via a monitoring server. There were pri- 
marily two client user interfaces to the monitoring 
server, namely the Dashboard Designer and various 
SharePoint Web Parts. The Dashboard Designer 
was a thick client application downloaded from the 
monitoring server, which allowed power users to 
e Create data source connections 
e Create views that use those data connections 
e Assemble the views in a dashboard 
e Deploy the dashboard to Microsoft Office 
SharePoint Server 2007 or Windows SharePoint 
Services (WSS) 


All of this information was stored to a SQL Server 
2005 database that was managed directly through the 
monitoring server. 

Once a dashboard had been published to the mon- 
itoring system database, it could then be deployed 
to Microsoft Office SharePoint Server 2007 or Win- 
dows SharePoint Services. Therefore, in that sense, 
PerformancePoint Server was a product that worked 
in parallel with SharePoint. Yet another portion of 
the PerformancePoint Server was the planning center 
operation. PerformancePoint Planning Server sup- 
ported a variety of management processes, which 
included the ability to define, modify, and maintain 
logical business models integrated with business 
rules, workflows, and enterprise data. Finally, there 
was the management report, which was a component 
designed for financial reporting. 
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That is all history! Microsoft Office Performance- 
Point Server was discontinued in April 2009. Before 
you feel sad or shed any tears for the loss, the product 
was instead reincarnated as PerformancePoint Services 
for SharePoint 2010. It is available as part of non-free 
versions of SharePoint 2010. It is envisioned to be quite 
disruptive to the marketplace that it is entering because it 
is a highly reengineered and well thought out product. 

What is PerformancePoint Services for SharePoint 
2010? PerformancePoint Services for SharePoint 2010 
is the part of SharePoint that allows you to create 
rich, context-driven dashboards that aggregate data 
and content to provide a complete view of how your 
business is performing at all levels. In other words, it 
is the easiest way to create and publish business intel- 
ligence dashboards in SharePoint 2010. At the heart 
of PerformancePoint Services is the Dash- 
board Designer. The Dashboard Designer 
is a thick client that you can launch directly 
from the browser, and it allows you to create 
KPIs, scorecards, analytic charts and grids, 
reports, filters, and dashboards. 

Compared to PerformancePoint Server 2007, there 
are many enhancements in PerformancePoint Services 
for SharePoint 2010. Some of these enhancements 
include the following: 

e Enterprise-level scalability: Built upon the new 
services infrastructure in SharePoint 2010, 
PerformancePoint Services has the ability to scale 
out more than PerformancePoint Server 2007. 

e SharePoint repository: There is no longer a 
separate monitoring server database. All objects 
that are created are now stored in the content 
database. This has numerous advantages centered 
on security, administration, backup and restore, 
and even the end-user experience. 

e All PerformancePoint features are now Share- 
Point features: There is a business intelligence 
repository available as a site definition or you 
have the ability to create new sites based upon 
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other site definitions and enable certain features 
to make use of PerformancePoint features in any 
site collection. 

e PerformancePoint filters can now be connected 
with standard SharePoint Web Parts, because 
they build upon the standard WSS Web Parts 
connection framework. 

e Integration with SharePoint also makes it 
possible for PerformancePoint to work with 
SharePoint features such as search, indexing, 
workflows, Excel Services, and Visio Services. 

e There are some significant improvements in the 
various SharePoint Web Parts, chart types, and 
the Dashboard Designer. 


With this theory, let’s start with the process of 
administrating, configuring, and using Performance- 
Point Services for SharePoint 2010. Configuring 
PerformancePoint Services is split into two halves: 
one that you need to do in Central Administration 
and one that you need to do in the site collections you 
intend to use with PerformancePoint Services. 


PerformancePoint Central 
Administration Settings 

If you used the Farm Configuration Wizard to 
configure your SharePoint installation, chances 
are PerformancePoint Services is ready to go for 
you. Let’s look at the specific configuration nec- 
essary to use PerformancePoint Services on any 
particular farm. Visit Central Administration for 


Edit PerformancePoint Service Application 


[28] Specify settings for this service application. You can change these settings later from the Manage Service Applications page. 


|Name: 


Specify a name and default status for this service 


| application 


your farm. Under Central Administration click on 
Manage service applications. PerformancePoint Ser- 
vices is yet another shared services application within 
SharePoint. Look for PerformancePoint service appli- 
cation within Central Administration; if one isn’t 
here, choose to create one in Central Administration 
using the page at /_admin/ServiceApplications.aspx. 
If you click on the properties for the Performance- 
Point service application, you should see a screen 
similar to Figure 1. 

The check box that you see next to the name of the 
PerformancePoint service application is telling Share- 
Point that all new web applications by default will 
use this particular instance of the PerformancePoint 
service application. Therefore, for any web site to use a 
different instance you would have to explicitly go into 
the web site settings and allow a certain web site to use 
a different application instance. This is an important 
consideration from a planning and scalability point 
of view. 

There is yet another very important service 
available on SharePoint called the Secure Store 
Service. The Secure Store Service is the evolution of 
what used to be single sign-on in SharePoint 2007. 
Single sign-on in SharePoint 2007, and the Secure 
Store Service, provides a secure mechanism to store 
various credentials for various application IDs. Spe- 
cifically, if the PerformancePoint service application 
was not associated with a Secure Store Service, you 
won't be able to set an unattended service account 
for PerformancePoint Services to connect to data 
sources. Ås a result, the only mecha- 
nism you can use to connect to data 
sources would be where the identity of 
the logged in user is used to connect to 


Help 


the back-end data source. This means 
that Kerberos must be running prop- 


erly on your network. For practical 
reasons though, you need to configure 


| The setting makes this service application available by i 
| default for web applications in this farm to use. Do not VV Add this service application's proxy to the farm's default proxy list. 


| check this setting if you wish to specify manually which 
web applications should use this service application. 


Application Pool 


Choose the Application Pool to use for this Service 
Application. This defines the account and credentials that will 
be used by this web service. 


You can choose an existing application pool or create a new 
one. 


Figure I 


PerformancePoint service application settings 
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Kerberos on your network anyway, so 
this is not such a big deal. 

Note that PerformancePoint Services 
is claims aware. However, in today's 
world, a lot of back-end data sources 
are not claims aware. As a result, even 
though the identity being passed within 
the PerformancePoint infrastructure is 
a claims-based identity, the identity 
required to talk to back-end data sources 
is usually a Kerberos identity. 

Next, choose to manage the 
PerformancePoint service application. 
You will see a screen with four options, 
as shown in Web Figure 1 (www.sglmag 
.com, InstantDoc ID 125181). 

Starting from the bottom, the Import 
PerformancePoint server 2007 content 
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option, as the name suggests, allows you to import 
content from previous versions of PerformancePoint 
to PerformancePoint Services for SharePoint 2010. 

Trusted content locations and trusted data source 
locations work in a manner very similar to Excel Ser- 
vices. By default, all SharePoint locations are trusted. 
This is a setting similar to Excel Services. 

The setting at the top, PerformancePoint service 
applications settings, is where all of the other set- 
tings go. At the top of this page is a section for 
Secure Store Service and the unattended service 
account. In Central Administration, under Manage 
service applications, if you use the Farm Configura- 
tion Wizard to set up your farm, you should see an 
instance of the Secure Store Service already created 
for you. If it hasn't been created, go ahead and 
create one. 

Back in the PerformancePoint service applica- 
tion settings page, provide the name of the Secure 
Store Service application name and provide an 
unattended service account. The unattended service 
account is what will be used to authenticate with 
back-end data sources. Thus, you want to ensure 
that this account is not a highly privileged account 
and is different from your farm account or any 
application pool accounts. Not doing so may inad- 
vertently give access to data sources that you didn't 
plan on giving access to. Also, before you're able to 
specify the unattended service account, you will first 
have to visit your Secure Store Service application 
settings page and ensure that you generate a new 
key first. This generated key requires you to specify 
a password. 

The one final setting, which you do not have to 
perform but you should know about, is that under 
Manage web applications, select your port 80 web 
application and choose Service Connections from 
the ribbon. This should pop open a dialog box 
that lets you associate various service application 
instances with existing web applications, as shown 
in Figure 2. 


PerformancePoint Site 
Collection Settings 

Let me start with a bit of good news first. If you have 
installed SharePoint Enterprise Edition, you do not 
need to do any more farm-level or web application- 
level configurations. The various Web Parts, web 
services, and the Dashboard Designer are already 
there for you. All you need to do now is activate the 
appropriate features on the sites and site collections 
to start using PerformancePoint Services. 

Right out of the box there is a site definition called 
the Business Intelligence Center provided for you, 
which makes use of all the necessary features. All the 
features that make up that site definition can also be 
individually activated in other site collections so that 
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you can use PerformancePoint Services anywhere you 
wish. This gives you immense flexibility. 

I will demonstrate the use and configuration of 
PerformancePoint Services in a blank site collection. 
Start by creating a blank site collection at the root level 
in your port 80 web application. Configuring your site 
collection is a matter of activating a few features. You 
need to activate the following site collection features in 
this order: 

1. SharePoint Server Enterprise Site Collection 
Features 

2. SharePoint Server Publishing Infrastructure 

3. PerformancePoint Services Site Collection 
Features 


Then, under Site Features (not Site Collection), 
activate the PerformancePoint Services Site Features 
feature. 

Note that SharePoint Server Publishing 
Infrastructure is a prerequisite for PerformancePoint 
Services site collection features. This is because the 
dashboard publishing uses the SharePoint Server 
Publishing Infrastructure. 


Configure Service Application Associations 


Edit the following group of connections: [default z] 


Name Type 


Access Services 


| 


Proxy 


Access Services Web Service Application 
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Application Registry Service 


Business Data Connectivity 
Service 


Excel Services Application 


Managed Metadata Service 


PerformancePoint Service 
Application 


PowerPoint Service 
Application 


Search Service Application 
Secure Store Service 
State Service 


Usage and Health data 
collection 


User Profile Service 
Application 


Visio Graphics Service 


Web Analytics Service 
Application 


Word Automation Services 


Word Viewing Service 


Application Registry Proxy 


Business Data Connectivity Service 
Application Proxy 


Excel Services Application Web Service 
Application Proxy 


Managed Metadata Service Connection 
PerformancePoint Service Application Proxy 
PowerPoint Service Application Proxy 
Search Service Application Proxy 

Secure Store Service Application Proxy 
State Service Proxy 


Usage and Health Data Collection Proxy 
User Profile Service Application Proxy 


Visio Graphics Service Application Proxy 


Web Analytics Service Application Proxy 


Word Automation Services Proxy 


Word Viewing Service Application Proxy 


ax | ^c | 


Figure 2 


Service application settings for a web application 
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Once these features are activated, activate the 
PerformancePoint Services Site Features at the site level. 


Using PerformancePoint 
PerformancePoint Server 2007 used a separate data- 
base to store all of its necessary information. That has 
changed in PerformancePoint Services, because all the 
necessary information is now stored inside of lists 
and document libraries; lists that contain dashboards, 
definitions, reports, scorecards, filters, KPIs, and 
indicators; as well as document libraries that contain 
exported dashboards and data sources. 

Now that you have properly configured 
PerformancePoint both in Central Administration 
and in the site collection, you have available all the 
list definitions and the necessary content types to get 
started. 

1. Create a new list based on the Performance- 
Point content list list definition and call it 
PerformancePoint Content. 

2. Create a new document library called 
Dashboards based on the Dashboards Library list 
definition. 
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3. Create another document library called Data 
Connections based on the Data Connections for Per- 
formance Point list definition. 


In the PerformancePoint content list, you should 
see all the necessary content types, as shown in 
Figure 3. 

One thing is clear: The driving forces behind 
PerformancePoint Services are content types. There- 
fore, anything that applies to content types, reusability, 
queryability, structure, and information management 
policies can be used with PerformancePoint content. 

Click on any of the content types and that 
will launch a click-once application, which is your 
Dashboard Designer. Once the Dashboard Designer 
is launched, you can then work entirely in the 
Dashboard Designer to create various artifacts. Start 
by saving your dashboard as MyDashBoard.ddwx 
on your local disk before you add any new items in 
the workspace. At this point, your workspace in the 
Dashboard Designer should look Figure 4. 

Before you start creating artifacts in 
PerformancePoint, first set up the database that you 
will use. PerformancePoint Services is very versatile 
and can work with various back-end sources such 
as Excel Services, SQL Server Analysis Services, 
a regular SQL Server table, or even SQL Server 
Reporting Services. I will demonstrate the usage 
of PerformancePoint Services with SQL Server 
Analysis Services. 

1. To begin, set up the AdventureWorks sample 
databases downloaded from msftdbprodsamples 
.codeplex.com/releases/view/24854. 

2. Once you have downloaded and installed the 
AdventureWorks sample databases, open the C:\ 
Program Files\Microsoft SQL Server\100\Tools\ 
Samples VAdventureWorks 2008R2 Analysis Services 
Project\enterprise\Adventure Works.sln project in 
the Business Intelligence Development Studio of 
SQL Server 2008. 

3. After opening the solution in the Solution 
Explorer, double-click the Adventure Works.ds 
data source. Click the Edit button just below the 
Connection String box. Supply your SQL Server 
(database engine) server and instance name (if 
it’s a named instance). Click the Test Connection 
Button. If the test succeeds, click OK to save the 
changes. 

4. Right-click the solution in Solution Explorer 
and choose Properties. On the Adventure Works 
DW 2008 SE Property Pages dialog box, choose 
the tree item for Deployment. Change the Target, 
Server property to your SQL Server Analysis 
Services server name and instance name (if it is not 
a default instance). 

5. In the Solution Explorer, right-click the solution 
(Solution ‘Adventure Works’) and click Deploy. 
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Once the project is deployed, right-click the data 
connections document library in your Dashboard 
Designer and choose to create a new data source. 
Create a new SQL Server Analysis Services-based 
project, using the Adventure Works DW 2008R2 
database and the Adventure Works cube, as Figure 5 
shows. Choose to save this data connection as 
AdventureWorks. 

Next, right-click the PerformancePoint content list 
and choose to add a new report. When prompted, 
choose to add a new analytic chart. Choose the new 
AdventureWorks data connection you have just created 
as the data source for this analytic chart. In the new 
report, you can drag and drop product categories 
in the series, and in the bottom axes you can drag 
and drop geography. The new report is shown in 
Figure 6. 

Save this report as AdventureWorks Products. 
Once this report is saved in your list, note that it 1s 
saved as a certain content type. I’d like to point out 
here that if you were to add that content type to any 
other list, you would be able to save your reports in 
any list in SharePoint you wished. This really gives 
you the maximum flexibility you need. 

Back on your SharePoint site on the default 
page, put the page in edit mode and drop the Per- 
formancePoint report Web Part. Then, point this 
Web Part to this newly created report that you've 
just uploaded in your SharePoint site. You should 
note that the report is running in full interactivity 
on the SharePoint site. You should see a big blue 
bar called All geographies. Go ahead and click on 
the big blue bar, and the report should update, 
giving you the various products available. This can 
be seen in Web Figure 2. 

Why is it that my bike sales are so high? I'm not 
asking this, but I know my business users will. When 
you see the bright red bar, right-click it and choose 
decomposition. This brings up a Decomposition 
Tree for the underlying data source, and it would 
immediately tell you that you are selling a lot of road 
bikes (shown in Web Figure 3). 

Why are you selling so many road bikes? As an 
analyst now, you can click on it and easily choose 
the dimension you wish to expand upon (see Web 
Figure 4). Let's expand on Geography, which reveals 
that you are selling a lot of bikes in California. 
Expanding further by city reveals that you are 
selling a lot of bikes in Carson, and especially in the 
months of May, November, August, and February. 
This is a pattern that repeats every year (as seen in 
Figure 7). 

This clearly tells you that the Carson market 
is pivotal to your company's success! And, you 
were able to determine this without bothering a 
developer. SQL 
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VARBINARY(MAX) 


Tames the RBR LO B 


Use this new data type to easily import and 


retrieve BL OB data 


Editor's Note: We recognize that many SQL Server 
Magazine readers are still running SQL Server 2005 
and SQL Server 2000 systems in their SQL Server 
environments. Therefore, we will be periodically resur- 
facing popular SQL Mag articles that focus on solu- 
tions for those versions. If there's an article that you 
would like to see us run again in SQL Mag, email 
mkeller@sqlmag.com. 


ike the jelly-like Blob monster in those 
L Steve McQueen horror movies, mixing 

binary large objects (BLOBs) with relational 
database data has traditionally been unnatural and 
difficult to manage. BLOBs and relational database 
data are very different entities. Relational data 
usually consists of text or numbers and tends to 
be small. In contrast, BLOB data is most often pic- 
tures in .jpg, .tiff, or .bmp format—such as product 
images on a website—which can be quite large. 

As the web-exclusive sidebar "Storing BLOBs 
in the Database or the File System?" (www.sqlmag 
.com, InstantDoc ID 92996) discusses, integrating 
BLOB data and relational database data has its pros and 
cons. And until recently, SQL Server's limited data types 
for storing BLOBs made working with them more dif- 
ficult than working with traditional relational database 
data. But with the introduction of three new data types, 
SQL Server 2005 treats all data the same—whether your 
BLOB contains images, large amounts of text data, 
audio files such as MP3s, or even program executables 
(.exes). In this article, you'll see how easy it is to use one 
of the new data types, VARBINARY(MAX), to import 
BLOB data into and retrieve it from a SQL Server 2005 
database. You can download a simple example project 
that demonstrates this functionality. But before jumping 
into the code, let's review the key elements of SQL 
Servers BLOB storage. 


What about BLOB? 

SQL Server stores standard INT, CHAR, and VAR- 
CHAR data directly within a row. However, this 
approach limits the maximum capacity of each data 
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type to 8,000 bytes, slightly less than SQL Server's 
maximum row size of 8,060 bytes. (Note that with SOL 
Server 2005's row-overflow feature, the maximum row 
size can exceed 8,060 in certain cases; for information 
about this feature, see Kalen Delaney's "Stretching the 
8K Row Limit,” InstantDoc ID 50490.) 

Although more than adequate for most data types, 
the 8KB limit is a problem for most BLOB data. To 
accommodate the needs of larger BLOB data, Micro- 
soft designed SQL Server to handle BLOB storage 
differently than it handles storage for more common 
data types. Figure 1 shows an overview of how SQL 
Server 2005 and earlier releases store the IMAGE and 
VARBINARY(MAX) BLOB data types. 

As you can see in Figure 1, SQL Server 
doesn't store large BLOB data on the same 
data page as the data for the rest of the row. 

Instead, it stores BLOB data as a collection 

of 8KB pages organized in a B-tree structure. 

Each row's BLOB column contains a 16-byte pointer to 
the root B-tree structure, which tracks the blocks of data 
that comprise the BLOB. If the data is less than 64 bytes, 
SQL Server stores it as part of the root structure. Oth- 
erwise, the root structure contains a series of pointers to 
the data blocks that comprise the binary object. 

For BLOBs smaller than SQL Servers 8KB 
page size, you have a couple of options for storing 
the BLOB data inline, as you would standard text 
and numeric data. For the old TEXT, NTEXT, and 
IMAGE data types, which SQL Server 2005 continues 
to support, you can use the fext-in-row feature to store 
the data inline. And for the new VARCHAR(MAX), 
NVARCHAR(MAX), and VARBINARY(MAX) 
data types, you can use the backward-sounding large 
value types out of row option. Storing smaller BLOB 
data inline improves performance, avoiding the extra 
I/O needed to read the BLOB data record. (For 
more information about SQL Server's text-in-row 
feature, see Kalen Delaney's “Text in Row Internals,” 
InstantDoc ID 37635.) 

Note that BLOBs are sometimes called large 
objects (LOBs) or character large objects (CLOBs). 


Michael Otey 


(motey @ sqlmag.com) is technical director 

for Windows IT Pro and SQL Server Magazine 
and author of Microsoft SQL Server 2008 New 
Features (Osborne/McGraw-Hill). 


MORE on the WEB 


Download the code at InstantDoc ID 
92995 and see the web-exclusive 
sidebar at InstantDoc ID 92996. 
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The LOB designation can refer to both CLOBs and 
BLOBs. However, technically, CLOBs can contain 
only text data such as XML, whereas BLOBs can 
contain any type of data, including binary data. 
SQL Server 2005 can store LOB data by using the 
older TEXT, NTEXT, and IMAGE data types or the 
new VARCHAR(MAX), NVARCHAR(MAX), and 
VARBINARY(MAX) data types. Table 1 lists the 
attributes for SQL Server's LOB data types. 

In pre- SQL Server 2005 releases, IMAGE is the 
primary data type for BLOB storage and can contain 
2GB of virtually any type of binary data. Although 
similar to IMAGE data types, TEXT and NTEXT 


TABLE l:Attributes of SQL Server's LOB Data Types 


TEXT 


Stores non-Unicode, 
variable-length text data 


2^31 — 1 (2,147,483,647) 
2GB 


Deprecated 


IMAGE Stores variable-length 2^31 — 1 (2,147,483,647) 


binary data 2GB 


Deprecated 


(SQL Server 2005, 
2000, 7.0, and 6.5) 


(SQL Server 2005, 


pr e VARBINARY(MAX) TAMES THE BLOB 


data types can store only text data, not binary data. 
TEXT can accommodate up to 2GB of non-Unicode 
text data, and NTEXT can accommodate up to IGB 
of Unicode text data. 

However, these earlier BLOB data types have 
some frustrating limitations. You can't use them 
as local variables in stored procedures or T-SQL 
batches, as part of an index, orin WHERE, ORDER 
BY, COMPUTE, or GROUP BY clauses. (The 
exception: You can use TEXT and NTEXT with the 
LIKE keyword.) However, the biggest problem with 
IMAGE, TEXT, and NTEXT data types is that to 
access them, you have to use a different programming 
model than you use to access other SQL Server data 
types. 

SQL Server 2005s VARCHAR(MAX), 
NVARCHAR(MAX), and VARBINARY(MAX) data 
types eliminate these limitations while still letting you 
store large amounts of data. Unlike with earlier BLOB 
data types, you access the various (MAX) data types the 
same way as other SQL Server data types. In addition, 
you can declare the new data types as local variables. 
And you can use SQL Server string-handling functions 
such as SUBSTRING() on VARCHAR(MAX) and 
NVARCHAR(MAX) columns. VARCHAR(MAX) 
and VARBINARY(MAX) both provide up to 2GB 
of storage, whereas NVARCHAR(MAX) provides 
1GB of character storage. Now, let's see how you can 
use the new VARBINARY(MAX) data type with 
T-SQL and ADO.NET to import BLOB data into 
a SQL Server 2005 database, then use ADO.NET to 
retrieve it. 


Using T-SQL to Import BLOBs 
The first step in working with BLOBs and the 
VARBINARY(MAX) data type is to create 
a table containing a VARBINARY(MAX) 
column and use the column as the target to 
import BLOB data. Use the following code to 
create the sample table MyBLOBTable: 


CREATE TABLE MyBLOBTable 
(blob id int IDENTITY(1,1), 
blob description varchar(256), 
blob data varbinary(max)) 


2000, 7.0, and 6.5) 


NVARCHAR(MAX) Stores Unicode, variable- 2430 — 1 (1,073,741,823) Current 
length text data 1GB of character storage (SQL Server 2005 
and later) 
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Because of T-SQL's limited ability to 
address the IMAGE, TEXT, and NTEXT 


LISTING I: Using T-SQL to Load 
VARBINARY(MAX) Data 


INSERT INTO 
MyBLOBTable (blob description, 
blob data) 
SELECT 'Windows Server', 
BulkColumn FROM Openrowset( 
Bulk 'C:\temp\Win2993.bmp', 
SINGLE BLOB) AS blob 
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data types, these older BLOB types almost forced you 
to use ADO or ADO.NET to load BLOB data. But 
because you can use the new (MAX) data types as 
local variables, you can either use T-SQL to load them 
directly or use ADO or ADO.NET to load them. 

Listing 1's code uses T-SQL and the OPENROW 
SET function to load data into the VARBINARY 
(MAX) blob data column. The INSERT statement 
inserts the results of the SELECT statement, which 
uses the OPENROWSET function to read the con- 
tents of the bitmap file win2003.bmp from the file 
system. Notice that OPENROWSET uses the new 
BULK rowset provider, which reads data from the 
file system, to return a file's contents as a scalar 
value. The SINGLE BLOB keyword tells the BULK 
provider that it will be reading binary data. 


Using ADO.NET 2.0 to Import 
BLOBs 

You can also use ADO.NET to import BLOB data 
into VARBINARY(MAX) columns. As I noted 
earlier, for the IMAGE data type, ADO.NET is the 
only effective way to load BLOB data. However, 
earlier versions of ADO.NET require a different 
programming model to work with BLOBs than to 
work with other character and numeric data types, 
which makes BLOB processing more difficult. In 
ADO.NET versions before 2.0, you essentially 
have to use either chucking techniques or Stream 
objects to access BLOB data. (See *BLOB Further 
Reading" at www.sqlmag.com, InstantDoc ID 
92995 for articles that discuss these techniques.) 
However, Microsoft has enhanced ADO.NET 2.0 
to support the new VARBINARY(MAX) data type, 
greatly simplifying BLOB handling. 

The code in Listing 2 uses ADO.NET 2.0 to 
load data into the blob_data column. First, the 
code reads the BLOB data from the file system into 
a variable. As callout A in Listing 2 shows, the code 
creates a new FileStream object named fs to read the 
file C:\tem\win2003.bmp". The fs FileStream object 
then reads the BLOB file’s contents into a byte array 
named bBLOBStorage. 

After the code reads the BLOB data from the file 
system and assigns it to a variable, it writes the vari- 
able’s contents to the SQL Server database. Because 
the code requires I/O to the file system and access 
to the database, you need to add the following 
namespaces to your project: 

e Imports System.IO 
e Imports System.Data 
e Imports System.Data.Sqlclient 


At callout B in Listing 2, the code creates a new 
ADO.NET SqlConnection object named cn and a 
new ADO.NET SglCommand object named cmd. 
As their names suggest, the cn SglConnection object 
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creates a connection to the SOL Server database, 
whereas the cmd SglCommand object executes a 
T-SQL INSERT command that adds the BLOB data 
to the database. In this example, the INSERT state- 
ment uses two parameters: (Qblob description to 
add a description of the BLOB to the database and 
(Øblob data to add the BLOB itself to the 
database. 

The code sets the Direction property of both 
parameters to input and sets the data type of the 


LISTING 2: Using ADO.NET 2.0 to Load 
VARBINARY(MAX) Data 


Sub ImportBLOB() 


(A) Dim fs As New FileStream 


C'c:\temp\win2993.bmp", FileMode.OpenOrCreate, FileAccess.Read) 


Dim iLen As Integer - CInt(fs.Length) 
Dim bBLOBStorage(iLen) As Byte 
fs.Read(bBLOBStorage, 0, iLen) 


fs.CloseO 
Dim cn As New SqlConnection(sConnectionString) 
(B) Dim cmd As SqlCommand = New SqlCommand _ 


C"INSERT INTO MyBLOBTable (blob description, blob data) " _ 


& "VALUES(Gblob description, 
cmd.CommandType - CommandType.Text 
cmd.Parameters.Add("Gblob description", SqlDbType.VarChar) 


Gblob data)", cn) 


Classics 


cmd.Parameters("Gblob description").Direction = ParameterDirection.Input 


cmd.Parameters.Add("Gblob data", SqlDbType.Image) 


cmd.Parameters("Gblob data").Direction = ParameterDirection. Input 


' Store the file name 

cmd.Parameters("Gblob description").Value = "My BLOB" 
' Store the byte array in the image field 
cmd.Parameters("Gblob data").Value - bBLOBStorage 


cn.Open() 
cmd. ExecuteNonQuery() 


End Sub 


LISTING 3: Using ADO.NET to Read 
VARBINARY(MAX) Data 


Sub RetrieveBLOB() 
Dim cn As New SqlConnection(sConnectionString) 
Dim da As New SqlDataAdapter 


("SELECT blob data FROM MyBLOBTable WHERE blob id = 1", cn) 


Dim cb As SqlCommandBuilder = New SqlCommandBui Ider (da) 
Dim ds As New DataSet 


Try 
cn.Open() 
da.Fill(ds, "MyBLOBTable") 
Dim rw As DataRow 
rw = ds.Tables("MyBLOBTable") .Rows (0) 


' Make sure you have some rows 
Dim i As Integer = ds.Tables("MyBLOBTable") .Rows.Count 
If i » 9 Then 
' Set up a byte array for the BLOB 
Dim bBLOBStorage() As Byte - 


ds.Tables("MyBLOBTable").Rows(i - 1)("blob data") 


' Create a memory stream for the transfer 
Dim ms As New MemoryStream(bBLOBStorage) 


PictureBoxl.SizeMode - PictureBoxSizeMode.StretchImage 


PictureBoxl.Image = Image.FromStream(ms) 


ms.Close() 
End If 


Catch ex As Exception 
MsgBox(ex.ToString) 


Finally 
cn.Close() 
cn = Nothing 

End Try 


End Sub 
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(Øblob description parameter to SqlDbType.VarChar 
and the data type of (Qblob data to SqlDbType 
Image. Don't be misled by the use of SgIDbType 
Image for (Qblob data; this data type isn’t restricted 
to just the IMAGE data type; it also works with 
VARBINARY(MAX). Next, the code assigns values 
to the parameters’ Value properties, assigning a string 
to (Qblob descriptions property and assigning the 
contents of the bBLOBStorage byte array, which 
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was filled earlier, to @blob_data’s property. The code 
opens the cn SglConnection object. Then, it uses the 
cmd SqlCommand object's ExecuteNonQuery method 
to execute the T-SQL INSERT statement, which 
imports the BLOB data to MyBLOBTable’s blob data 
column. 


Retrieving Data from a 
VARBINARY(MAX) Column 
After you've loaded BLOB data into the database, 
you need to be able to retrieve and display the 
data. The code in Listing 3 uses ADO.NET 2.0 
to retrieve the BLOB data from MyBLOBTable’s 
VARBINARY(MAX) blob data column and dis- 
play it in a picture box. In this example, the code 
reads the BLOB into a MemoryStream, then assigns 
the MemoryStream to a picture box control's Image 
property. This technique is fast and doesn't require 
any intermediate files, eliminating the I/O required 
to first write the binary data to disk, then read it. 

Listing 3s RetrieveBLOB subroutine begins at 
callout A by creating a new SqlConnection object 
named cn and a new SqlDataAdapter object named 
da. The code constructs the SqlDataAdapter by 
using a T-SQL SELECT statement that retrieves the 
blob data column from MyBLOBTable, where the 
value of the blob id column equals 1. The code then 
creates a SqlCommandBuilder object and an empty 
data set. A Try block opens the connection and uses 
the SqlDataAdapter’s Fill method to populate the 
data set. The subroutine then evaluates the MyBLOB- 
Table DataTable’s Rows.Count property to make sure 
some data was retrieved. 

The real action begins at callout B with the cre- 
ation of a byte array called bBLOBStorage, which 
is assigned the contents of the binary image in the 
DataSet’s blob_data column. The code then creates a 
new MemoryStream object named ms and assigns it 
the contents of the bBLOBStorage byte array. Finally, 
the code uses the picture box control’s FromStream 
method to assign the binary image data from the ms 
Memory-Stream object to the picture box control's 
Image property. Figure 2 shows the results of the 
RetrieveBLOB subroutine. 


BLOB with Ease 
Although SQL Server 2005 still supports the 
TEXT, NTEXT, and IMAGE data types for 
backward compatibility, the VARCHAR(MAX), 
NVARCHAR(MAX), and VARBINARY(MAX) 
data types give you important capabilities for working 
with BLOBs. As you've seen in these examples using 
VARBINARY(MAX), you can now use these data 
types as local variables and in T-SQL parameters, so 
you can easily work with BLOB data in T-SQL and 
ADO.NET 2.0. E 
InstantDoc ID 92995 
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he Common Language Runtime (CLR) is at 
E the core of the .NET Framework develop- 
ment environment. It is the part of .NET that 
is ultimately responsible for memory management, 
thread management, exception handling, and security. 
The CLR has not had any significant changes since 
2005 when .NET Framework 2.0 was shipped. All the 
version 3 frameworks that shipped in the meantime 
(.NET 3.0, 3.5, and 3.5 SP1) didn't modify the CLR in 
a significant way. But CLR 4.0 adds some significant 
enhancements in every area of the CLR. 


Side-by-Side Execution 

A key starting point in understanding the improve- 
ments in CLR 4.0 is side-by-side execution. Folks that 
remember the pain of migrating from 1.1 to 2.0 will 
rejoice: Your applications are not forced to roll forward 
to the newest framework any more. Applications built 
in 2.0 or newer versions of the framework will continue 
to run on that version of the framework and CLR 
without incident, even within the same process. 

This becomes hugely important when you're using 
third-party libraries and tools in your application. You 
can move your own code up to 4.0, but the components 
can still be running on 2.0. This feature alone mitigates 
alot of fear of upgrading to the latest and greatest; you 
don't have to make sure every bit of your application is 
migrated to 4.0 all at once. 

It also has significant impact if youre still using 
COM components. In previous versions of the CLR, the 
COM component had to run in the same version of the 
framework as the application itself. With the 4.0 CLR, 
COM components can run under their native framework 
version (typically version 2), or you can tell CLR 4.0 to 
support 2.0 behavior if the 2.0 framework isn't available 
by modifying the startup tag in the configuration section 
of the application's configuration file, like this: 


«configuration» 
«startup useLegacyV2RuntimeActivationPolicy= 
"true"» 
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«supportedRuntime version="v4.Ø" /» 
«/startup» 
«/configuration» 


Also, in the application configuration file you can 
specify preferred frameworks in sequence. So if you 
prefer your application run on .NET 2.0 SP1, but 
that version isn't available to run on .NET 4.0, the file 
would look like this: 


«configuration» 
«startup » 
«supportedRuntime version="v2.Ø.5727" /> 
«supportedRuntime version="v4.Ø" /» 
«/startup» 
«/configuration» 


For many situations, you need to make no changes to 
your application to move to the 4.0 CLR and frame- 
work. The various components of your application will 
use the versions of the framework that it requires. 


Garbage Collection 

Most developers take the memory management 
abilities of the CLR entirely for granted. You never 
need to think about how variables get allocated in 
memory; you just declare them. And you don't worry 
much about cleaning them up—it just happens. The 
CLR makes memory management almost entirely 
transparent. 

The only aspect of CLR memory management 
that developers are aware of is garbage collection. In 
the CLR, variables are allocated quickly, but freed up 
slowly. Deallocating a variable, typically by letting it fall 
out of scope, marks the variable as deleted but does not 
return the memory to the heap for use—that happens 
only when garbage collection runs. For the most part, 
garbage collection is invisible to the application. In fact, 
it is strongly recommended that you don't mess with 
garbage collection, such as forcing a collection to run 
with a GC.Collect statement. 
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The only time most developers think about 
garbage collection is when it affects their application. 
Typically this only occurs in two scenarios: client 
application freezes and page-processing delays on 
a web server. To understand how this happens, you 
need to understand a bit more about how garbage 
collection works. 

Memory management in the CLR is broken into 
three generations made up of groups of memory 
segments. The first two are Gen 0 and Gen 1, called 
ephemeral generations since they are generally very 
short-lived. When an object is first created, it lives in 
Gen 0 until garbage collection runs. If by the time 
garbage collection runs the object is dead, that is, 
already deallocated and finalized, the memory that 
was allocated to that object is freed for use. If the 
object is still in use, it gets promoted to Gen 1. In 
some cases an object is deallocated but not finalized; 
that is, the garbage collector will fire the finalizer of 
the object but still ships the object over to Gen I. 
The object will be cleaned up on the next garbage 
collection. 

If when garbage collection 1s run on a Gen 1 seg- 
ment there are objects still in use, those objects are 
moved to Gen 2. In reality, “move” is a misnomer—the 
garbage collector does its best not to move memory 
around, since doing so is a slow process. The garbage 
collector will compact down live objects to fill in the 
gaps in memory left by dead objects. But ultimately a 
segment normally never moves, it just gets marked as 
a different generation. 

Gen 2 segments are where the pain lives for 
garbage collection. Objects in Gen 2 tend to be long- 
lived and in applications where garbage collection 
becomes a problem—inevitably it is a large Gen 2 store 
that's the problem. Normally, garbage collection is a 
blocking process. This isn't a big deal for Gen 0 and 
Gen 1 segments because the garbage collection runs so 
quickly. But because Gen 2 can get so large, garbage 
collection takes longer, causing pauses in the UI of an 
application or forcing a web server to delay processing 
of web pages. 

Before CLR 4.0, garbage collection on Gen 2 
worked in concurrent mode. This meant that 
memory could still be allocated in Gen 0 while a 
Gen 2 garbage collection was running. So in effect, 
execution of your application is not interrupted 
by a Gen 2 garbage collection, at least for a little 
while. Where things break down is when it's time 
for a Gen 0 or Gen 1 garbage collection and the 
Gen 2 garbage collection is still running—then 
everything stops. The ephemeral generations cannot 
run garbage collection while the Gen 2 garbage 
collection is running. 

In CLR 4.0, background garbage collection 
has been introduced, replacing concurrent garbage 
collection. Background garbage collection allows the 


ephemeral generations to run simultaneously with 
the Gen 2 garbage collection. Actually, it's not truly 
simultaneous; what has been added is that while a 
Gen 2 garbage collection is running, it checks to 
see whether the Gen 0 or Gen 1 collection needs to 
run and pauses its processing to let the much faster 
ephemeral generation garbage collection finish. This 
should keep your application working as long as your 
threads aren't dependent on memory objects in the 
Gen 2 segment. 

One caveat is that this new background garbage 
collection is available only in the workstation mode 
of garbage collection. Server mode does not have 
this capability. Workstation and server mode gar- 
bage collection have nothing to do with the OS per 
se; they are modes that depend on multiple core 
machines. Server mode garbage collection will use 
many threads across many cores to do garbage col- 
lection very rapidly, although in the concurrent and 
blocking way. 

The workstation mode of garbage collection is 
the default mode of the CLR unless you're running 
in the CLR in ASP.NET or within SQL Server— 
then you default to server mode. You can also set 
your application to run in server mode using the 
<gcServer> tag in the application configuration file. 
Also, a single CPU machine always runs the CLR in 
workstation mode. 

Background garbage collection should address 
your applications having UI hauls due to garbage col- 
lection, but the workstation mode restriction means it 
won't help much for ASP.NET. In reality, background 
collection doesn't make sense when there are huge 
numbers of discrete processes all running against the 
same set of heaps, which is the normal condition of a 
busy ASP.NET site. 

But there's another feature in CLR 4.0 that a 
developer very concerned about ASP.NET garbage 
collection performance can take advantage of: garbage 
collection notifications. There's a "death spiral” that an 
ASP.NET server can go into when it's running short 
of memory and under extreme load. As memory gets 
low, the Gen 2 garbage collection runs to try and free 
up memory. Often threads are blocked because they 
depend on objects in Gen 2, like in-process session 
objects (you're not still using in-process session objects 
are you?). While the Gen 2 garbage collection runs, the 
server ends up with all threads blocked, resulting in 
request queues growing. When the garbage collection 
finishes, the server is hammered by all the requests 
that are backed up in the queue. That backlog runs the 
server out of memory again, causing another Gen 2 
garbage collection. With each iteration the queues get 
larger, until the worker process recycles or the server 
crashes. 

In CLR 4.0 you can subscribe to an event called 
RegisterForFullGCNotification, which notifies you 
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when a Gen 2 garbage collection will run. When this 
event occurs, you could set a flag that would cause 
all web pages to fire a Server. Transfer to a different 
server. Alternatively, you could send a command to 
a load balancer to effectively pull the machine having 
the garbage collection out of the server farm. 

While the redirect is running, monitor for the 
WaitForFullGCComplete to know when to stop doing 
redirection. This technique won't solve the workload 
already on your server, but it will stop additional traffic 
from making the situation worse. Note also that gar- 
bage collection notifications really only work in server 
garbage collection mode. You can make it work in 
workstation mode if you disable concurrent garbage 
collection. 


Thread Management 

Most of the changes in thread management for CLR 
4.0 have to do with the reality of parallel execu- 
tion. Multicore computers are the norm today, and 
programming has to adjust to take advantage of 
that. More cores naturally mean more threads in an 
application. 

Microsoft has added a number of libraries to take 
advantage of parallelism, including the Parallel Task 
Library, which is a topic for another article. (For an in- 
depth look at parallelism, see “Parallel Computing with 
.NET 4.0,” www.devproconnections.com, InstantDoc 
ID 125038.) The changes to the CLR 4.0 for thread 
management focus on dealing with many more threads 
in an application. A key change is the introduction of 
local queue for each thread in a process. 

Normally there's a single global work queue for 
all threads in a given application. As the number of 
threads grows, this queue would get very busy and 
contentious. In CLR 4.0, every thread gets its own 
local queue as well, largely to mitigate contention for 
work. The only problem with multiple queues is that 
you'll get to a situation where there are several items 
in one thread's local queue, but nothing in another 
thread's local queue or the global queue. CLR 4.0 
has a feature called work stealing, where a thread 
with nothing to do can "steal" work from another 
local queue. 

Like most features of the CLR, these new thread 
management capabilities are largely invisible to 
the average developer. But without them, the new 
styles of parallel development would be significantly 
impaired. 


Security 

Until CLR 4.0, security in .NET meant Code Access 
Security (CAS). CAS still exists in CLR 4, but it has 
changed significantly. CAS breaks down into three 
parts: policy, permissions, and enforcement. In the 
CLR 4, CAS policy has been deprecated. Policy deci- 
sions are left up to the host of the application. CAS 
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policy is where the pain of CAS largely lived: Caspol 
.exe was never easy to use, deploying policies is difficult, 
and the differences in behavior between managed and 
native code often confused things. 

What this means to the average application is that 
If a user starts the application directly (double-clicking 
an icon, for example), that application runs in full 
trust mode. After all, the act of the user starting the 
application is an implication of trust—you trusted it 
enough to start it. So why shouldn't it just work? 

That being said, the permissions and enforcement 
models of CAS are alive and well in CLR 4.0. Medium 
trust continues to work as per normal for security 
scenarios like ClickOnce and ASP.NET. Microsoft 
has also added the SecAnnotate tool to help expose 
libraries in a partial trust sandbox. 


Diagnostics and Performance 
One of the challenges developers have had in the past 
is determining whether a particular application domain 
was affecting other application domains. OS tools like 
Windows Task Manager monitor only at the process 
level. With CLR 4.0, processor and memory usage is 
measured per application domain. 

For the most part, the improvements in diagnostics 
come from CLR 4.0 adding additional events in Event 
Tracing for Windows (ETW). Specifically, there are 
now Application Domain Resource Monitoring events 
for memory and thread allocation at the application 
domain level. If you want to start doing detailed 
tracing of ETW, you should download the Windows 
Performance Analysis Tools at msdn.microsoft.com/ 
en-us/performance/cc825801 .aspx. 

There are many more features in CLR 4.0, par- 
ticularly core language elements expanding support for 
generics like System.Collections.Generic.SortedSet(T). 
There's also covariance and contravariance, a topic that 
could take a whole article to explain. CLR 4.0 also 
includes new data types, including System. Numerics 
.BigInteger (an arbitrary-precision integer), System 
.Numeric.Complex (for arithmetic and trigonometric 
operations) and a System.Tuple class for handling 
structured data. 


Moving Forward 
The latest version of the CLR is a big step forward, 
not only in dealing with past issues that made devel- 
opment challenging in some cases, but also setting the 
stage for new development techniques in the future. 
The addition of the Dynamic Language Runtime to 
support Ruby and Python, plus the addition of the F# 
language, indicates a few different directions in which 
Microsoft will be taking development in the future. 
Make sure you take some time to explore MSDN and 
other articles in DevProConnections magazine about 
these new features in the .NET 4.0 Framework. ET 
InstantDoc ID 124996 
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No Budget for Travel? No Problem! 
Get the training you need right at your desk with 


eLearning Courses 


http://elearning.left-brain.com 


Join industry experts for informative eLearning courses. 
Each course includes in-depth sessions as well as live Q&A. 


Our eLearning Series provides you with in-depth training 
on a variety of topics ranging from: 


Windows 7 

SQL 

Visual Studio 
NET 

SharePoint 

And Much More! 


Visit http://elearning.left-brain.com and view all our available 
classes. You can attend live or view a past course on-demand. 


Don't miss this opportunity for the training you need from the 
comfort of your own computer. 


Check out the eLearning Series offerings Is today! y 


NET Component Products 


Ease your programming burden 


he Microsoft .NET Framework has been 
i around for years—the company first 
announced it back in 2000. It makes the 
process of creating applications smoother for SQL 
Server developers—and just about any other kind 
of developer. Because .NET was designed from the 
ground up to make different programming languages 
and Microsoft products get along, it's an important 
product for SQL developers. If there's something you 
need .NET to do, but the functions you're looking for 
just aren't around, you can take advantage of .NET's 
ability to bring different products together. 

Innovations in the recent versions of Visual Studio 
have made it easier to access your databases from 
within applications, but there's always some other 
function that a developer could wish for. That's where 
.NET components come in. The products listed in 
this month's buyer's guide table are just a sample of 
what's available. Because .NET has so many uses, some 
products listed may be only peripherally related to SQL 
Server. On the same note, some vendors chose not to 
participate in the buyer's guide because they didn't feel 
SQL Server was a primary use of their products, but 
SQL Server developers could still find them useful. 
Always do your own research, and if you don’t see 
a product that does what you need on the list, keep 
looking. 

Even the relatively small list here includes 
products that do very different things. Some of 
the products use visual UI controls; others don't. 
One component helps you create Windows Forms. 
Another assists in making visualizations, such as 
charts and graphs. A third targets creation of sched- 
ules. The bottom line is that there's a huge variety of 
.NET component products out there, so if you need 
something and you can't find it, you're probably not 
looking hard enough. 

Now that .NET 4.0 and Visual Studio (VS) 2010 
are available, you have one more factor to consider 
when you're looking at .NET components. VS 2010 
brings many new features to the table over VS 2008, 
such as refined LINQ-to-SQL, an improved code- 
writing environment (with better multimonitor sup- 
port), and database-specific functions. See "Visual 
Studio 2010 for Database Professionals," InstantDoc 
ID 103679, for more on what's new in VS 2010 and 
.NET 4.0. 
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Many of the products in the buyer's guide table 
don't support .NET 4.0. It's possible to use VS 2010 
to create applications for older .NET platforms, but 
be careful. If you're on the cutting edge and you want 
to take advantage of Microsoft's latest developer offer- 
ings, you'll have to deal with your component options 
being cut. Vendors are usually pretty quick to upgrade 
their products to be compatible with Microsoft's latest 
versions, but check with a specific vendor before com- 
mitting to them. 

In addition to making sure you get component 
products that work with your version of VS and the 
.NET framework, make sure to look at what other 
technologies the components work with. In many 


v 
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Innovations in the recent 
versions of Visual Studio 
have made it easier to 
access your databases 
from within applications, 
but there's always some 
other function that a 


developer could wish 
for. That's where .NET 
components come in. 


areas of modern computing, the list of supported tech- 
nologies looks pretty much the same for all products 
of a certain time. That's not the case here—support for 
ASP.NET, AJAX, and mobile controls varies greatly 
from one product to another. 

If youre interested in learning more about 
developing with .NET, I recommend checking out 
our sister publication, DevProConnections (devpro 
connections.com). This publication, formerly known as 
asp.netP RO, has a large archive of articles on the . NET 
environment, VS, and databases. [SQL | 

InstantDoc ID 125307 


Qu Note 


Information in this buyers 
guide comes from vendor 
representatives and resources 
and is meant to jump-start, 
not replace, your own research; 
also, some products might 

have been left out, either as 
an oversight or from lack of 
vendor response. 
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Product WinForms ASP.NET AJAX 
Support Support Support 
ComponentArt ComponentArt Web.UI for ASP.NET $799 for No Yes Yes 
416-622-2923 Developer license; 
866-392-1175 $999 for 1-year 
www.componentart.com subscription 
license 
ComponentOne ComponentOne Studio for Silverlight Starting at No No No 
412-681-4343 $800 


800-858-2739 
www.componentone.com 


ComponentOne Studio for ASP.NET AJAX Starting at $800 No Yes Yes 
ComponentOne Studio for WinForms Starting at $800 Yes No No 
ComponentOne Studio Enterprise Starting at Yes Yes Yes 
$1,300 

Dart Communications PowerSNMP for .NET Starting at Yes No No 

315-339-8040 $1,399 

www.dart.com 
PowerTCP Zip Compression for .NET Starting at $299 Yes No No 
PowerTCP Telnet for .NET Starting at $249 — Yes No No 
PowerTCP SSH and SFTP for .NET Starting at $349 Yes No No 
PowerTCP SSL Sockets for .NET Starting at $499 Yes No No 
PowerTCP Sockets for .NET Starting at $249 Yes No No 
PowerTCP Email Validation for .NET Starting at $149 ^ Yes No No 
PowerTCP Mail for .NET Starting at $349 Yes No No 
PowerTCP FTP for .NET Starting at $299 Yes No No 
PowerTCP Emulation for .NET Starting at $699 Yes No No 

Infragistics NetAdvantage for .NET Starting at Yes Yes Yes 

609-448-2000 $1,195 

800-231-8588 

www.infragistics.com 

Klik! Software Klik! Windows Forms Library $799; discounts Yes No No 

+90 216 5537432 available 

www.kliksoft.com for multiple 

purchases 

TMS Software TMS Silverlight Planner No No No 

www.tmssoftware.com 
TMS ASP.NET Component Pack No Yes Yes 
TMS ASP.NET WebPlanner No Yes Yes 
TMS Flexcel Studio for .NET Yes Yes No 
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Mobile Controls | Ul Controls Custom Skin Programmable .NET Framework Version Supported 
Support Support Frameworks 
Support 
Yes No Yes Yes Yes 24059152210 
No No Yes Yes No 3.0 and higher 
No No Yes Yes No 2.0 to 4.0 
No No Yes No No 2.0 to 4.0 
No Yes Yes Yes No WinForms and ASP.NET AJAX (2.0 to 4.0), WPF 


and Silverlight (3.0 to 4.0), Mobile (2.0 and 3.0), 
ActiveX (Visual Basic 6) 


No No No No No 9x2 

No No No No No 3.5, 2.0, 1.1, 1.0 
No No No No No 3.8), 210) 

No No No No No Sep 210), LI 1040) 
No No No No No Sep 2) LIG 104) 
No No No No No 3.5, 2.0, 1.1, 1.0 
No No No No No 5, 200), LA 10) 
No No No No No 3.5, 2.0, 1.1, 1.0 
No No No No No She 20), ILL 1040) 
No No No No No 3153210 

No Yes Yes Yes Yes ASP.NET and Windows Forms UI controls: 4.0, 


3.5, 3.0, 2.0; WPF UI controls: 4.0, 3.5, 3.0; 
Silverlight UI controls for Line of Business 
Applications support Silverlight 4 


No No Yes Yes No 4.0, 3.X, 2.0 
Yes No No No No AL 

No No No No No ÅÅ Shey, 2 
Yes No No No No AL 39, Å 2 
No Yes No No No ALS Å 2 
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| can make my life a lot 


Automatically 
Shut Down 
and Start Up 
Services 


y poor workstation has to put up with some 
m unusual requirements for flexibility. Most of the 
time, it's a simple personal workstation I use to write 
documents, compile spreadsheets, and answer email. At 
other times, I need it to behave like a high-end database 
server, or at least act as high end as it can go. And in 
yet another set of situations, I need to push it to the 
utter limits of its capabilities while performing database 
benchmarks and tests. 

One of the things that I've found is that I can make 
my life a lot easier when switching from one workload to 
the other by dynamically enabling and disabling various 
Windows services that are running on the workstation. For 
example, I hardly need all of those services that Apple's 
iTunes require while I'm running a TPC benchmark over- 
night, but I really enjoy iTunes while I’m clicking away at the 
keyboard. My answer to this problem (and there are many 
other ways to do it, such as PowerShell) is to use CMD files 
to start and stop services as I need them. The CMD files can 
make use of two key utilities: SC.exe or NET.exe. 

For example, you might add a number of lines to 
shut down various Windows services using the syntax 
of either command, as in: 


SC STOP "iPod Service" 


or 


easier when switching from 
one workload to the other pevice" 


by dynamically enabling and 


NET STOP "Apple Mobile 


You could similarly write a 


disabling various Windows CMD file to start the same 


Windows services by substi- 


services that are running uting the STOP keyword with 
on th e worl stati on the START keyword. It doesn't 
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get much easier than that! 

I spent a few minutes 
parsing through my list of running services and, once I'd 
established all of the *user comfort" services, created the 
CMD file to shut them all down. I then created a second 
version of the file that started the same services, in case I 
wanted to switch from benchmarking mode to user mode. 

(Those are my terms, not industry standard terms). 
What other techniques have you used to automate pro- 
cesses like these? You can chime in on this and other topics at 
my blog at sqlmag.com/go/ToolTime. SOLI 


Savv Assistants 


Your guide to sponsored resources 


Training from Your Desk 
with eLearning On-Demand 
We bring the experts direct to you to share their 
real-world perspective, experience, and expertise. 
During each event, three sessions stream to allow 
you to get solutions. Check out our on-demand 
topics including SQL Server, Windows 7, Active 
Directory, and much more! 
windowsitpro.com/go/eLearning/OnDemand 


Programming SharePoint 
Business Connectivity 
Services 

Learn to create line of business applications that 
integrate external data sources with the SharePoint 
platform. SharePoint 20105 Business Connectivity 
Services make it easier than ever to build complex 
line of business applications based on the 
SharePoint platform. 
sharepointproconnections.com/go/SharePoint_ 
eLearning_June2010 


Gain Perspective Into Your 


Network 


Free Trial Download—Perspective is a compre- 
hensive and affordable network management and 
application monitoring solution. Solve problems 
with bandwidth, connectivity, network and appli- 
cation performance quickly and easily. It includes 
real-time performance monitoring, traffic analysis, 
virtualization support, advanced alerting, network 
mapping, and VolP support. Give it a try today. 
sqlmag.com/go/ trials/NetworkPerspective 


Follow us on Twitter atwww.twi 
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9 Editor's Tip 


Got a great 
new product? 
Send announce- 


BUSINESS INTELLIGENCE 
NovaView Expands Microsoft PowerPivot 
Panorama Software announced NovaView for PowerPivot, a business intelligence (BI) suite built on top 
of the Microsoft PowerPivot platform. Panorama NovaView for PowerPivot adds capabilities to the 
PowerPivot platform to increase usability and feature breadth. Self-service BI capabilities of the product Ments to products @ 
include a web interface, advanced analytics, and a data-level security layer. NovaView for PowerPivot sqlmag.com. 

also enables additional analysis, such as smart exceptions, instant calculations, and data formatting. — Brian Reinholz, 
Additional reporting features include pixel perfect reports and executive dashboards. To learn more, visit production editor 
www.panorama.com. 


VIRTUALIZATION 

Manage Backup Files Through 
SSMS 

Idera announced SQL virtual database 1.3, 
the latest release of a solution that allows 
backup files to look and act like a SQL A rid 
Server database within SQL Server Manage- || : backup 

ment Studio and other SQL Server appli- 
cations. This lets users run queries, insert 
and delete data, run reports, and perform 
resource-intensive checks against SQL Server 
database backup files. The latest version Å Backup Fle Fuon} \\EFTBEG9\Backups\Southwind, Full 200909181545 safe 
offers support for compressed backup files, CC HSULLIVANT 

support for instances located on the nodes access the det] VilualDalabase Name: (Southwind. Thursday. VDB 

of a cluster, and support for the latest ver- 
sions of SQL Server and Windows Server. 
SQL virtual database is available for $495 per 
server. To learn more, visit www.idera.com. 


It's that easy! 


SECURITY 

DBWall Provides Row-Level Security 
BiTKOO has released a new solution called DBWall that offers row-level security. DBWall acts like a form of 
firewall between user requests and the SQL Server database, ensuring that all requests are legitimate and safe 
before opening the door to the data. According to the vendor, key benefits of using DBWall include prevention 
of SQL injection attacks, consistent authorization and 
security controls (reducing manual effort), and user action 
auditing. To learn more, visit www.bitkoo.com. 


LIS gor ee Da mow se 
6 = 


DATABASE ADMINISTRATION 
Bradmark Updates Monitoring Suite 
Bradmark Technologies announces SurveillanceDB 4.3, 


a suite that offers real-time monitoring, issue identifica- eal ": €—À nov 
tion and resolution, rules-based event management, ines SEN prm aic 
data archiving, and reporting. According to the vendor, um — 
benefits gained from the product include automated EE 
msn 

event management, increased resolution efficiency, alert ae 

å : i Ke i fenomen 
and event notification, and resource usage identification. Seas tte 


New features in the latest release include: a redesigned 
GUI for heterogeneous database monitoring, support for 
the latest versions of SQL Server and Windows Server, 
and more. To learn more or try a free demo, visit www 
.bradmark.com. E 
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"38 SQL Server 2008 R2 


Express Installation 


Options 


s you might expect, when Microsoft released 

SQL Server 2008 R2, it also released new ver- 
sions of its free low-end edition, SQL Server 2008 R2 
Express. The RTM release date for SQL Server 2008 R2 
was officially April 21, 2010, a bit earlier than the other 
more full-featured editions of SQL Server 2008 R2. 

Microsoft made some important new changes 
in SQL Server 2008 R2 Express. One of the biggest 
changes is that the maximum database size has been 
bumped up from 4GB to 10GB. You might wonder 
if that means the maximum RAM size might have 
changed as well. However, it hasn't. SQL Server 2008 
R2 is still limited to IGB of RAM. 

The new SQL Server 2008 R2 Express product is 
packaged differently than previous versions of SQL 
Server Express. Instead of being labeled as a separate 
edition, each different SQL Server 2008 R2 Express 
package is now presented as an installation option. You 
can download the different SQL Server 2008 R2 Express 
editions at www.microsoft.com/express/Database/default 
.aspxtt Installation Options. Below, I guide you through 
the new options of SQL Server 2008 R2 Express so 
you'll have a better idea of what each offers. 


SQL Server 2008 R2 Express 
Database Only 

Tipping the scales at 71 MB for the x86 version and 
74MB for the x64 version, Database Only is the smallest 
and simplest installation option for SQL Server 2008 
R2 Express. This option is suitable for including in ISV 
packages and as a database for custom applications. 
This includes the core SQL Server 2008 R2 relational 
database engine but not the business intelligence (BI) 
subsystems like Analysis Services, Integration Services, 
or Reporting Services. This minimal installation option 
also doesn't include any built-in management tools. 
However, most of the SOL Server core features are 
included such as support for the CLR and support 
for all data types, including the new geospatial and 
FILESTREAM data types. 


SQL Server 2008 R2 Express 
Management Tools 

Be aware that the SQL Server 2008 R2 Express Man- 
agement Tools version doesn't include the relational 
database engine. This installation option includes only 
SQL Server Management Studio Express (SSMSE). 
The Management Tools are 154MB for the x86 ver- 
sion and 163MB for the x64 version. (Doesn't it seem 
somehow wrong that SSMSE by itself is almost twice 
the size of the relational database engine?) Anyway, 
you would use this option to manage previously 
installed instances of SQL Server Express. 


SQL Server 2008 R2 Express 
Database with Management 
Tools 

This version, SQL Server 2008 R2 Express Data- 
base with Management Tools, is probably the best 
version for low-cost database development or for 
use with small business and departmental database 
needs. It has all the relational database features 
plus it also has SSMSE, which you can use for 
basic database management. This version weighs 
in at 235MB for the x86 version and 247MB for 
the x64 version. 


SQL Server 2008 R2 Express 
Database with Advanced 
Services 
The SQL Server 2008 R2 Express Database with 
Advanced Services option is the most full-featured 
SQL Server 2008 R2 Express edition. It comes in at 
a hefty 727MB for the x86 version and 795MB for 
the x64 version. This installation option includes 
the basic SQL Server relational database engine, as 
well as SSMSE, support for full-text search, and 
support for Reporting Services. This Express version 
of Reporting Services works only with the local SQL 
Server 2008 R2 Express instance. E 
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WHAT ARENT YOU SEEING? 


SOL SENTRY CAN IDENTIFY PROBLEMS BEFORE THEY SNEAK UP ON YOU 


Download the free trial at sqlsentry.net 


SQL Sentry products are made by DBAs, for DBAs. They provide unparalleled insight, awareness and 
control over your SQL Server environment. With features like real-time and historical performance analysis 
and visual schedule management, SQL Sentry is the ultimate monitoring solution. 


SOLSENTRYN 


Find databases in need of a back up * Search SQL code for text strings * Search for objects 
across all SQL Servers e° Find rogue SQL Servers « Check password strength * Diagnose 
connection problems * Copy server logins + Run queries across multiple SQL Servers 
e Move databases * Clone users * Copy jobs * View and edit jobs * Rebuild indexes e 


Compare server configurations «f. ompare dasslasese ig nerate new 
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ects across all SQL Servers * Find rogue SQL Servers * 
Check password strength * Copy server logins * Diagnose connection problems * 
Run queries across multiple SQL Servers « Move databases * Clone users * Copy jobs 
e Rebuild indexes ° View and edit jobs « Compare server configurations e Compare 
database configurations * Generate new partitions « View existing partitions « Generate 
connection strings « Compress backups up to 95% « Collect SQL Server version info * 
Collect server hardware configurations * Analyze index usage and size * Find servers that 
need patches or hotfixes « Analyze SQL Server space usage * View and compare 
performance statistics « Recover table data from backups * Receive alerts when SQL 
Servers are unavailable * Report from backup data * Compare database schema * Synch 
database schema * Compare tables * Synch data * Capture snapshots of SQL schema e 
Automate schema and data synchronization * Cut backup times up to 50% 


SQL toolbox- 


Must-have tools for DBAs and Developers 


See how SQL toolbox can save you hours each day! 


GE) WATCH THE VIDEO 


= TOUR NOW! iclere 


idera.com/ToolboxVideo TOOLS FOR SQL SERVER 
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